A comparison of statistical models used to rank schools for accountability purposes
MetadataShow full item record
Public school accountability has become an important part of national educational policy. Schools are ranked for accountability purposes using a variety of statistical models. These models include performance, or status models, and productivity, or student change models. Performance models do not separate the influence of school effects from background effects such as socioeconomic status on student achievement, while productivity models can isolate school and background influences on student achievement. The current study investigated the differences between school rankings calculated using a performance model and three types of productivity models in terms of consistency of rankings and relation between ranking and school percent low socioeconomic status students. In the first study, using real data, school rankings were calculated using the percent passing, cohort difference, unadjusted and adjusted singlelevel regression, and unadjusted and adjusted multilevel models. A simulation study was also conducted which simulated 250 schools with varying percentages of low socioeconomic status students. Student achievement for 30 students was simulated with varying degrees of relation between school percent low socioeconomic status students and student achievement, student socioeconomic status and test score, and the amount of variation in student test scores between schools. School rankings were remarkably different when calculated using the different models; especially the percent passing model. The magnitude of differences is especially important when policy makers consider rewards for top-performing schools or sanctions for low-performing schools. Correlation of rank calculated using the percent passing model with school percent low socioeconomic status students was as high as 0.41 The simulation study showed that ranking calculated using each of the models was most highly correlated with school percent low socioeconomic status students when there was a strong (-0.10) simulated relation between student socioeconomic status and individual student test score as well as a relation (-0.10) between percent low socioeconomic status students and individual student test score. In all conditions the correlation between rank and school percent low socioeconomic status students was weaker when a larger proportion of variance in student test scores was within school.