Methods for assessing student learning in the State of Arizona

Abstract

The effectiveness of a method using scaled scores and a correction for regression to the mean (RTM) designed to measure academic growth attributable to schools was compared to several alternative methods all incorporating simple (unadjusted) growth. Problems with scaled scores and the correction for RTM were discussed. Three alternative methods using normal curve equivalent (NCE), percentile rank (PR), and stanine scores were presented and compared to the scaled score method. A variation of the scaled score method without the correction for RTM was proposed to examine the effects of the correction. Two variations of the NCE and PR score methods were constructed with adjusted passing criteria to examine the effect of accounting for measurement error. Matched-student (1998--1999) Stanford 9 Achievement Test scores from the State of Arizona were used to compute a dichotomous one year's growth indicator (OYG) and a five-point within-state rank-ordered growth indicator (the Star Rating) for each school/grade unit using each of the proposed methods. Results showed that the methods using NCE or PR scores were more likely than the method using scaled scores to assign the same OYG decision to each school/grade unit. The correction for RTM resulted in school/grade units with low initial status having to (inappropriately) make more than one year's worth of growth to achieve a passing OYG decision. The results tended to confirm correlations between initial status and the simple growth indicators in the alternative methods, but for a majority, the magnitudes of the correlations were not large enough to warrant dismissing simple growth. Recommendations from the study were: (1) Scaled scores and the correction for RTM should not be used in any of the methods; (2) Methods that account for error should be used to allow for control over the possibility of misidentification of failing schools as well as the proportion of schools that are identified as needing assistance; (3) The current minimum unit size criterion of eight students should remain, because increasing the number would result in too many units not included in analyses.