A GUIDE TO DETERMINING SAMPLE SIZE A PRIORI INTRODUCTION A priori sample size determination is a complex task in particularly when the exact population size is unknown. It requires a set of priori assumptions that most of the time is not known to market researchers with absolute accuracy. It is also specific to the type of statistical methodology1 that the research will aim to utilise to answer unique market research questions2. It also most often than not is dependent on the type of specifics of the statistical methodology. For instance, for a logistic regression model, the research question maybe to estimate the sample size necessary to achieve, in a two-sided test with a power of at least 0.95. Then the knowledge that the researcher need to (a priori) have are the distribution of the population and the odds ratio3. The following discussion is based on estimating the adequate sample size a priori for a number of regression methods. The most commonly used regression types by Roy Morgan are Multiple Linear Regression, Logistic Regression, Multinomial Logistics Regression and Ordinal Regression. MULTIPLE LINEAR REGRESSION Linear regressions are the most commonly used regression techniques in the industry. Therefore there is a large body of work in the literature when it comes to estimating a priori sample sizes in relation to linear regressions. Brooks, G. and Barcikowski, R. (2012) point two distinct approaches to estimating a priori sample size for multiple linear regressions. 1 T-test (means/proprtions), Odds ratio, Chi-Square and Contingency tables, ANOVA, ANCOVA, MANOVA, MANCOVA, Correlation, Regression, Discriminant Function Analysis and Cox Regression 2 Dattalo, P. (2008) 3 Hsieh et al. (1998) 1 Conventional Rules Conventions have evolved that are based on the premise that with a large enough ratio of subjects to predictors, the sample regression coefficients will be reliable and will closely estimate the true population values4. For example, Stevens (2002) suggested a ratio of 15 subjects for each predictor (i.e., with 7 predictors 105 subjects are required) and Pedhazur and Schmelkin (1991) recommended N>30k, where k is the number of predictors5. These rules lack any measure of effect size, therefore they can only be effective at specific— usually unknown—effect sizes. For example, a 15:1 subject-to-predictor ratio is acceptable only if the population squared multiple correlation is moderately large (i.e., over .40); otherwise, as the true squared multiple correlation decreases, expected cross-validity reduces (Brooks, 1998). Statistical Power Methods6 From a statistical power perspective, Multiple Linear Regression provides several alternative statistical significance tests that can be the basis for sample size selection7. These methods incorporate the effect size8 as well as the statistical power parameters into the equation. REGRESSION METHODS BASED ON ESTIMATING ODDS Regression methods based on estimating Odds (i.e. Logistic, Multinomial Logistic and Ordinal Regressions) require larger sample sizes than the simpler linear regression methods. This is due to the fact that maximum likelihood estimates are less accurate than ordinary least squares (i.e., simple linear and multiple linear regression models)9. For instance, to achieve a 95% statistical power in a multiple regression method with less than 10 independent variables requires about 100 cases whereas to reach the same statistical power in a logistic regression requires close to 350 cases10. 4 Miller & Kunce (1973), Pedhazur & Schmelkin (1991), Tabachnick & Fidell (2001) Others have provided rules that combine some minimum value with a subject-to-predictor ratio, including N>30+10k (Knapp & Campbell-Heider, 1989),N>50+k (Harris, 1985), and N>50+8k (Green, 1991) 6 Statistical power is the probability of rejecting the null hypothesis when the null hypothesis is indeed false 7 Cohen (1988), Cohen & Cohen (1983), Green (1991), Kraemer & Thiemann (1987), Milton (1986) 8 Standardised mean difference between the two groups 9 Hsieh, F.Y. et al. (1998) 10 Author’s own calculations based on Demidenko (2007), Hsieh et al. (1998) 5 2 The example here is only for illustrative purposes and it should not be taken as a standard guide. Each model requires a unique treatment of its own depending on the ultimate goal of the model, the properties of the data and the desired specifications of its statistical power. REFERENCES Brooks, G. and Barcikowski, R. (2012) Multiple Linear Regression Viewpoints, Vol. 38(2). Brooks, G. P. (1998). Precision Efficacy Analysis for Regression. Paper presented at the meeting of the Mid-Western Educational Research Association, Chicago, IL. Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences, (2nd Edition), Lawrence Earlbaum Associates, Hillsdale, NJ. Cohen, J. & Cohen, P. (1983) Applied multiple regression/correlation analysis for the behavioural sciences (2nd ed.), Hillsdale, NJ: Erlbaum. Cohen, J., Cohen, P., West, S.G., and Aiken, L.S. (2003) Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (3rd edition), Lawrence Earlbaum Associates, Mahwah, NJ. Dattalo, P. (2008) Determining Sample Size: Balancing Power, Precision, and Practicality, New York: Oxford University Press. Demidenko E. (2007) Sample size determination for logistic regression revisited, Statistics in Medicine 26:3385-3397. Green, S. B. (1991) How many subjects does it take to do a regression analysis? Multivariate Behavioral Research, 26, 499-510. Harris, R. J. (1985) A primer of multivariate statistics (2nd ed.), Orlando, FL: Academic Press. Hsieh, F.Y., Block, D.A., and Larsen, M.D. (1998) A Simple Method of Sample Size Calculation for Linear and Logistic Regression, Statistics in Medicine, Volume 17, pages 1623-1634. Knapp, T. R., & Campbell-Heider, N. (1989) Numbers of observations and variables in multivariate analyses, Western Journal of Nursing Research, 11, 634-641. Kraemer, H. C., & Thiemann, S. (1987) How many subjects? Statistical power analysis in research, Newbury Park, CA: Sage. Miller, D. E., & Kunce, J. T. (1973) Prediction and statistical overkill revisited, Measurement and Evaluation in Guidance, 6(3), 157-163. Milton, S. (1986) A sample size formula for multiple regression studies, Public Opinion Quarterly, 50, 112-118. 3 Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates. Stevens, J. (2002) Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Tabachnick, B. G., & Fidell, L. S. (2001) Using multivariate statistics (4th ed.), Boston: Allyn and Bacon. 4