Research and Development Survey: 2004

Survey background

The Research and Development Survey (R&D Survey) 2004 was run by Statistics New Zealand and sponsored by the Ministry of Research, Science and Technology (MoRST).

The R&D Survey measures the level of research and development activity, employment and expenditure by private sector enterprises, government departments, government-owned trading entities, and universities. The R&D Survey is carried out biennially by Statistics New Zealand. Results from surveys prior to 2002 have been released by MoRST.

This is the first release of data from the 2004 R&D Survey. A full report will be released in July 2005.

Changes to the R&D Survey 2004

Statistics New Zealand and the Ministry of Research, Science and Technology have used the 2004 survey as a chance to review the methodology as well as evaluate the outputs of the survey. Consequently a number of changes have occurred to the R&D Survey 2004 from when it was last run in 2002.

The survey was redeveloped to achieve the following key objectives:
1. Improved coverage of R&D performers in the economy
2. Collect more reliable and meaningful data
3. Produce standardised and repeatable methodologies

Questionnaires

In 2002 there was a university survey, designed specifically for capturing university R&D data, and also a general R&D survey which captured information from all other enterprises.

The 2004 survey utilised four different sector-specific questionnaires in order to capture information on R&D in a more efficient manner. Questionnaires were specifically designed for the following sectors:

• Business
• Government
• Crown Research Institutes (CRIs)
• Universities

These questionnaires were designed in a way that enabled the results to be compiled together to form summary statistics.

However, due to the nature of some of the changes made to the questionnaires, users should be careful when making comparisons with previous data. The following are a list of key changes to the questionnaires.

Differences between the questionnaires

As mentioned above, there were three types of form used to collect data from the business, government and CRI sectors. The differences between the business and government sector questionnaires are as follows:
1. The definition of R&D used is tailored towards each sector.
2. The 'own funds' definition is tailored towards each sector.
3. The business sector form groups the 'pure basic research' and 'targeted basic research' into one category ('basic research'). There are also differences in the ordering of the type of research categories.

Population methodology

In previous years the R&D Survey has been a census of all known R&D performers in the New Zealand economy. The populations were selected from various government agency funding lists and also from a Statistics New Zealand R&D indicator from the Annual Frame Update Survey (AFUS).

This census methodology in the past has now changed to a sampling methodology. This methodology change was conducted to develop more accurate statistics of R&D performed in New Zealand. This method will allow improved comparability of results between survey periods, and will eliminate the reliance on lists from other agencies.

More details on the new methodology can be found under 'Target population' later in the Technical Notes.

Comparisons with 2002 R&D data

By changing from a census methodology to a sample we are now introducing a degree of sampling error. More information on sampling error will be provided later in the Technical Notes.

Due to the re-ordering of the types of research question for the business R&D questionnaire, it is likely that the data will show a difference for the private sector specifically. It is believed that the results captured in the 2004 survey provide a more accurate portrayal of the types of research being carried out by enterprises in New Zealand. This is likely to have the same effect for the question on source of funds.

As a result, caution needs to be taken when making comparisons between the data from the R&D Survey 2004 and previous R&D Surveys.

Data collection

The R&D Survey was a postal survey utilising four questionnaires; a business form, a government form, a crown research institute (CRI) form, and a university form. These forms were specifically designed to capture data on R&D from these different sectors in the most efficient manner.

The universities questionnaire was designed to allow the universities to use financial information that is generally produced for the purposes of annual reporting. This means a number of data items for universities’ R&D were produced using modelled information. The New Zealand Vice Chancellors’ Committee (NZVCC) and MoRST assisted Statistics New Zealand in the questionnaire development, and in the determination of the modelling specifications.

The business, government and CRI R&D surveys were posted out in late August 2004. Information collected included the number of personnel within an enterprise working on R&D, current and capital expenditure on R&D, expenditure by type of R&D, source of funds for R&D carried out, and the area of application of the R&D. Data was requested where possible for the financial year ending 30 June 2004, or the last financial year within the 12 months up to 30 June 2004.

The R&D Universities Survey was posted out in March 2005 and data was collected for the year ended the 31 December 2004. Information collected included university discretionary income, internal and external research funding, academic staff salaries, university operating expenditure by faculty, and R&D personnel data.

Target population

Enterprises were included in the R&D Survey (business, government and CRI) population if they satisfied the following conditions:

• Economically significant (GST sales greater than $30,000) on the Statistics New Zealand Business Frame
• Not classified to ANZSIC codes 'D', 'G', 'H', 'I' or 'N'
• Included in one of the following categories:
  1. Had 'Yes' as the research and development indicator in the Annual Frame Update Survey (AFUS); or
  2. Had 'No' as the research and development indicator in the Annual Frame Update Survey (AFUS) and had a Rolling Mean Employment (RME) greater than or equal to 2, and was included in tiers 1 or 2 in the Statistics New Zealand Business Frame (tier 1 and 2 on the Statistics New Zealand Business Frame include enterprises with GST turnover of greater than $200,000); or
  3. Was included in a list of enterprises having a significant contribution to total expenditure on R&D, or to R&D funded by the enterprise but carried out by others in the 2002 R&D Survey, but not satisfying the previous criteria.

Note: The exclusion of ANZSIC codes 'D' (Electricity, Gas and Water Supply) and 'H' (Accommodation, Cafes and Restaurants) is due to the fact that these industries performed no R&D activity according to the results of the 2002 R&D Survey. The contribution of industries 'G' (Retail Trade) and 'I' (Transport and Storage) to the total reported for the Total Expenditure on R&D was 0.06% and 0.03%, respectively. Such contribution was considered too small to justify their inclusion in the survey population.

Sample design

The R&D Survey uses a stratified sample in its sample design. Strata were developed based on industries defined by ANZSIC.

Substrata were then developed using the following variables:

1. Whether an enterprise had a 'Yes' or 'No' to the R&D indicator on the Statistics New Zealand Business Frame. This indicator is captured from the Annual Frame Update Survey (AFUS).
2. The rolling mean employment of the enterprise from the Statistics New Zealand Business Frame. This indicator is captured from tax data.
3. Annual GST sales of the enterprise from the Statistics New Zealand Business Frame. This indicator is captured from tax data.

These substrata were sampled in three different ways . Firstly, they could be selected for full coverage, meaning that all enterprises in the substratum were selected for the survey. Secondly, they could be selected as forced full coverage, which meant that there was 100 percent coverage of firms seen as key to the survey, while all other enterprises in the substratum were randomly sampled. Thirdly, the entire sample selection for the substratum could have been selected using sampling.

Sampling error

The sampling error on the total R&D expenditure figure has been measured at 7.2 percent at the 95 percent confidence level.

Sampling error on the private sector total R&D has been measured at 12.8 percent. This in turn means that the true value of R&D of the private sector could fit between $565 million and $731 million.

Sample error on the government sector total R&D has been measured at 16.2 percent. There is no sampling error on the university sector.

Measurement errors

The R&D Survey results are subject to measurement errors. These need to be considered when analysing the results from the survey.

Measurement errors include mistakes by respondents when completing the questionnaire, variation in respondents’ interpretation of the questions asked, and errors made during the processing of the data. In addition, the survey applies imputation methodologies to cope with non-respondents and item non-response. See later in the Technical Notes for more information on imputation.

Statistics New Zealand adopts procedures to minimise these types of error, but they may still occur and are not quantifiable.

Given the nature of the data collected, there are limitations on the level of accuracy that can be expected from the R&D Survey. Many respondents do not keep separate account of their R&D expenditure, or they may include R&D with other scientific and technological services, such as consulting.

Response rate

The target overall response rate for the R&D Survey 2004 was 85 percent for business / government / CRIs. The survey achieved an actual response rate of 89 percent.

The target population for the R&D Survey consisted of 2,495 enterprises, plus the eight universities. The target overall response rate for the R&D universities survey was 100 percent, which was achieved.

Analysis of results

The R&D survey results have been compared with annual reports and other indicators published by Statistics New Zealand. Where the survey results differed substantially, a more detailed study of the data was made.

Imputation methodology

The following gives an outline of the imputation methodology used in the R&D Survey (business / government / CRI). No unit non-response treatment was required for the R&D universities survey, as a 100 percent response rate was achieved.

Unit non-response

Unit (or complete) non-response occurs where units in the population do not return the questionnaire, or an invalid questionnaire is received. A weight adjustment method is used to rate up the responding firms to compensate for the non-responding firms within the same unit non-response estimation cell. The responding firms are multiplied by the inverse of the response rate for the estimation cell.

For responding firms in estimation cell i:
1. Identify the responding firms that are outliers in estimation cell i. These firms are 'unlinked' and not used in estimation for non-response.

2. Assign a ‘non-response adjustment factor’ or weight (NONRESFACTi) to the responding firms within estimation cell i. 

Estimates of the total are produced by summing the weighted values of the responding firms.
((NONRESFACTij * yj ) is the weighted value of firm j in estimation cell i for variable of interest y).

Item non-response

Item (or partial) non-response is where units return the questionnaire but fail to provide breakdowns for selected aggregates.

Item non-response was applied to those breakdowns where a total could be sourced from another question. The item non-response method then used the mean proportion of all responding linked units within the item non-response estimation cell, and applies these proportions to the sourced total.

Published sector

Government sector (excluding universities)

All enterprises with the following New Zealand Institutional Sector 1996 codes:

Private sector

All enterprises with the following New Zealand Institutional Sector 1996 codes:

Universities

The university sector includes the eight New Zealand universities which are members of NZVCC. These are classified to New Zealand Institutional Sector 1996 code of 3111 (Cent Govt excl Funded Social Security), with an Australian and New Zealand Standard Industrial Classification – NZ Version 1996 code of N843100 (Higher Education).

Published industries

The published industries within the sectors detailed above include:

Government sector (excluding universities)

Private sector

University sector

Total universities.

Definitions

ANZSIC: Australian and New Zealand Standard Industrial Classification System – NZ Version 1996.

Statistics New Zealand Business Frame: A register of all businesses operating in New Zealand.

Enterprise: A legal business entity operating in New Zealand.

Research and development (R&D): The definition of R&D used in this survey is consistent with the OECD recommendations contained in the Frascati Manual. R&D performed by enterprises is generally investigative work which is of actual or potential use in the development of new or enhanced materials, products, devices, processes or services. R&D directed towards duplicating work already developed by others is only included if the knowledge or technology required for the development is not available to the enterprise.

Basic research: is carried out for the advancement of knowledge, without seeking long-term economic or social benefits or making any effort to apply the results to sectors responsible for their application.

Applied research: is also investigation undertaken in order to acquire new knowledge. It is, however, directed primarily towards a specific practical aim or objective.

Experimental development: is systematic work, drawing on knowledge gained from research and practical experience, that is directed at producing new materials, products and devices; installing new processes, systems and services; or improving substantially those already produced or installed.

Copyright

Information obtained from Statistics New Zealand may be freely used, reproduced, or quoted unless otherwise specified. In all cases Statistics New Zealand must be acknowledged as the source.

Liability

While care has been used in processing, analysing and extracting information, Statistics New Zealand gives no warranty that the information supplied is free from error. Statistics New Zealand shall not be liable for any loss suffered through the use, directly or indirectly, of any information, product or service.

Timing

Timed statistical releases are delivered using postal and electronic services provided by third parties. Delivery of these releases may be delayed by circumstances outside the control of Statistics New Zealand. Statistics New Zealand accepts no responsibility for any such delays.

Next release...

Research and Development in New Zealand: 2006 will be released on 17 September 2007.