Assessing sample size and mobility limits for a two-level multiple membership random effects model

Abstract

The purpose of the present study was to examine the minimal sample sizes and mobility rates needed for accurate estimation with the multiple-membership random effects model (MM-REM), and the conditions in which the MM-REM provided improved estimation over a traditional multilevel model (MLM). The mobility rate, level one and level two sample size, and the ICC for the level one predictor were manipulated for both a traditional MLM and an MM-REM. Relative parameter bias, relative standard error bias, and credible interval coverage were evaluated across 36 conditions for both methods. Standard error estimates of the intercept were negatively biased and credible interval coverage was low for both methods, and estimation improved as the level one sample size increased and as the ICC for the level one predictor decreased. Additionally, for MLM estimates, standard error bias decreased and credible interval coverage improved as the mobility rate decreased. Negative relative parameter bias was found for estimates of the level two coefficient, which was found to increase as the mobility rate increased for both methods. The level two variance component was overestimated with the MM-REM and underestimated with the MLM, and credible interval coverage was low for both methods. Estimation improved for MM-REM estimates as the level two sample size increased, and as the mobility rate decreased for MLM estimates. The results from the study suggest that, if applied researchers are interested primarily in estimates of the regression coefficients associated with predictors, both the MLM and the MM-REM provide accurate estimates of the level one coefficient, and accurate credible intervals for estimates of the level two coefficient. When applied researchers are interested in the variance components, however, the MM-REM should be used over the MLM when mobility exceeds 10% and the level two sample size is 40 or greater. The results of the study for each condition, in addition to study limitations and recommendations for applied researchers, are discussed.