An Examination of the Instructional Validity of the Arizona Instrument to Measure Standards

Abstract

The dissertation describes a study of the instructional validity of the Arizona Instrument to Measure Standards (AIMS), a standards-based assessment. The study addresses the third- and fifth-grade mathematics portion of the 2005 AIMS test, focusing on two performance objectives per grade level. The study centers on the following questions: Can variation in students' mathematics achievement on AIMS be explained by instructional validity measures, namely: (1) alignment between test items and instructional characteristics and (2) by the degree of teacher emphasis on the two skills of interest to the study? Does the relationship between these measures and AIMS performance differ across grade levels? What possible explanations exist to account for grade level differences? Is there a relationship between the instructional validity measures and performance on the objectives of interest to this study?The dissertation discusses the evolution of thinking about instructional validity as standardized testing has changed. The study methods, including developing alignment measures from interview transcripts and classroom assessment examples collected from 16 third-grade teachers and 20 fifth-grade teachers in one school district are also described. Findings include that, although the method of using qualitative data to gauge instructional validity yields rich information about instructional practice, there was little instructional variation between classrooms in the district studied. This may have occurred because the district requires teachers to provide instruction exactly as specified in the district-adopted mathematics text. Some between-grade level differences do exist in the curricular alignment with AIMS. Teachers attempted to overcome this in their instruction despite district mandates to the contrary. Results support the instructional sensitivity of AIMS at third grade, but not at fifth grade. Differences in instructional sensitivity across grade levels might be linked to curricular differences; some third-grade teachers reported supplementing the curriculum to address the state standards while fifth-grade teachers largely reported that this was not necessary. Interestingly, the degree of alignment at third- and fifth-grade did not vary, although fifth-grade teachers placed more emphasis the study objectives. This speaks to third-grade teacher commitment to address the standards, and the challenges in emphasizing them when district-adopted curricula are not well-aligned with state standards.