Assessing Language Assessment Literacy in University and College-Administered Intensive English Programs Across the United States

Abstract

Accountability and instructional decisions based on language assessment results should be derived from sound assessment practices carried out by assessment literate professionals. In the past forty years, researchers in mainstream education and more recently in the language assessment field have recognized that assessment illiteracy is widespread, with assessment being an undervalued and overlooked area of both pre-service training and professional development on the job. The current project investigated whether this was the case in the context of university and college-affiliated Intensive English Programs (IEPs) in the United States. Gathering the perspectives of two key stakeholder groups (IEP program directors and ESL teachers), this study explored perceived levels of language assessment literacy, attitudes toward language assessment, and preparedness to assess student learning and to use assessment results effectively and reliably. It also unveiled participants’ motivations to become more assessment literate and the challenges that are encountered along the way. Data for this mixed-methods study was collected via online questionnaires (n = 135) and through semi-structured follow-up interviews (n = 39). Most program directors and most teachers reported that they were knowledgeable to extremely knowledgeable in language assessment, and that they perceived this knowledge to be sufficient. Nonetheless, the majority of participants in both groups reported having received only basic training in language assessment, even though most had taken a stand-alone assessment course in their master’s program. While the assessment landscape in graduate programs appears to be improving, the focus is still highly psychometric-based and not directly applicable to the day-to-day needs of language teachers and program directors, who then transition to professional roles not equipped with the theoretical and practical knowledge needed to effectively perform their functions. Directors and teachers overall expressed a need to receive further training, but such training must be targeted to where one is in their development of language assessment literacy to be effective. Collaborative efforts were perceived to be the most beneficial and less threatening approach. While flawed assessments and meaningless grades still exist in the context studied, ESL teachers and program directors seem to be increasingly committed to understanding assessment as an essential component of teaching and to promoting best practices. By identifying the gaps in assessment knowledge and stakeholder needs, this study offers recommendations to support more robust professional development. It also responds to a call in the field to further delineate differential levels of assessment literacy and proposes profiles for these two groups of stakeholders who have been largely understudied in the literature.