Sample sizes may vary enormously. The smallest quantitative surveys comprise less than 100 interviews (often 60 is cited as the smallest ‘stable’ sample); while the largest surveys comprise many thousands of interviews. In deciding on the sample size, we need to take into account a number of factors…
- The size of the ‘population’ of interest - for example, if we are talking to chefs in upmarket restaurants, there may only be a few hundred in the whole of New Zealand, so our survey sample may be as small as 40 or 60 interviews. On the other hand, if we are looking at the general population, we would need to interview several hundred people to be sure that we have a broad range of people. Note however, that population size ceases to be a consideration where the population is large - a sample representing the population of a small town could realistically be the same size as a sample representing the whole country - with similar statistical validity.
- The number of subgroups which we want to analyse. Here we consider issues such as how many brands there are in the market, since we may want to analyse several different brands’ customer groups separately. The sample must then be big enough to give us a stable base of customers for each of these brands.
- The way the information will be used. Here we consider the level of ‘error’ which can be tolerated, and what level of accuracy is required for the Client to make the right decision. Some surveys have more rigorous needs than others in this regard, and since sample size is the biggest determinant in the cost of quantitative research, we try to match the sample to the needs.
- Whether or not the information will be used ‘in public’. Surveys which are open to public scrutiny - such as those conducted for City Councils or Government Departments - tend to require bigger samples simply in order to be ‘credible’ in the eyes of the general population. Although the statistical margin of error varies very little, a survey of 800 people is much more ‘publicly credible’ than one of 400.
The margin of error on a survey statistic is calculated to reflect the desired level of confidence required, usually 95% confidence in New Zealand studies.An indication of the statistical margin of error applying to various samples is shown below, at 95% and 90% confidence respectively.
These margins apply when the figure being measured is around 50%, and decline for smaller and larger figures.To interpret these figures, consider the margin of error of +5.7% at 95% confidence, on a sample of 300 interviews. This means that if we took 100 different samples of 300 people, from the same population, we would find that 95 of these samples would yield a result within +5.7% of the average, while 90 of the samples would fall within +4.7% of the mean.
Maximum Statistical Margin of Error
|
Sample Size |
At 95% confidence |
At 90% confidence |
|
60 100 200 300 400 500 600 700 800 900 1000 |
+12.7% +9.8% +6.9% +5.7% +4.9% +4.4% +4.0% +3.7% +3.5% +3.3% +3.1% |
+10.6% +8.2% +5.8% +4.7% +4.1% +3.7% +3.3% +3.1% +2.9% +2.7% +2.6% |
At 95% confidence, the margin or error on a survey statistics, such as P% of the population do this, is calculated as follows….
(Where… N=population size and n= sample size)
The margin is greatest (‘maximum margin of error’) when P is 50% - ie. The survey result shows that 50% of people think a particular way. So, for a population of 300,000 people, the maximum margin of error of a sample of 300 people is….
The above example shows why the formula is generally shortened to…
… since the factors relating to population size approach 1 with large populations.
In small groups, however, the situation is different. For example, if the defined population is 25 City Councillors, a sample of 15 would have a much smaller error margin than one might expect, as shown below….
The key point is that you only need to think about the size of the sample if the population is SMALL and / or you are planning to survey a large proportion of them.