Assessment Literacy: A Study of EFL Teachers’ Assessment Knowledge, Perspectives, and Classroom Behaviors
AuthorAl Bahlani, Sara Mohammed
ESL/ EFL Assessment
Language Assessment practices
Teacher Training on Assessment
MetadataShow full item record
PublisherThe University of Arizona.
RightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction, presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
EmbargoRelease after 12/01/2019
AbstractTeacher assessment literacy is essential to the success of teaching (T. H. Wang et al., 2008), the quality of student learning (DeLuca, Klinger, Searle, & Shulha, 2010; Mertler, 2004; White, 2009) and student learning motivation (Alkharusi, 2013; Dorman & Knightley, 2006). However, studies have consistently shown inadequate levels of assessment literacy for both pre-service and in-service teachers in language teaching and general education (Alkharusi, 2011; Malone, 2013; Mertler & Campbel1, 2005; Plake & Impara, 1992). This dissertation research aims to understand the current state of EFL teacher assessment literacy in six colleges of applied sciences in Oman by investigating teacher self-perceived language assessment literacy (competence and frequency of practice) as well as assessment knowledge and practice in the classroom. Its theoretical framework are Brookhart’s (2011) contemporary conceptualizations of formative assessment, Eyal’s (2012) discussion of digital assessment literacy as an important component in measuring teacher assessment literacy and Alkharusi’s (2009; 2010) methodological approach to investigating assessment literacy. This study implements a mixed-method approach using a triangulation of five data sources including self-assessment surveys, a language assessment knowledge test, an assessment evaluation task, classroom observations with a focus on teacher-produced assessment tasks, and teacher interviews. Multivariate analyses were used to examine relations between assessment literacy and participant background characteristics, such as gender, academic preparation, teaching experience, experience as coordinators, pre-service training, and in-service training. Research findings point to strengths and weaknesses in EFL teacher assessment literacy as well as matches and mismatches between teachers’ self-perceived assessment literacy and demonstrated assessment knowledge. Overall EFL teachers view themselves as moderately competent in language assessment. The variable with the strongest effect on teacher’s assessment literacy was pre-service training in assessment. The study discusses implications of research findings and offers suggestions for EFL teacher preparation programs, teaching institutions, and future research.
Degree ProgramGraduate College
Second Language Acquisition & Teaching
Degree GrantorUniversity of Arizona
Showing items related by title, author, creator and subject.
DECISION SUPPORT FOR RAPID ASSESSMENT OF TRUTH AND DECEPTION USING AUTOMATED ASSESSMENT TECHNOLOGIES AND KIOSK-BASED EMBODIED CONVERSATIONAL AGENTSNunamaker, Jr., Jay F.; Patton, Mark; Nunamaker, Jr., Jay F.; Burgoon, Judee K.; Zeng, Daniel; Koput, Ken (The University of Arizona., 2009)A pressing need exists for effective decision support systems to facilitate the rapid and accurate screening of large volumes of people. Millions of travelers transit through international borders and secure areas on an annual basis. Humans are exceptionally poor at detecting lying and deception and perform, on average, no better than chance. This research study focuses on the development, design and implementation of a Kiosk for Rapid Assessment of Deception (K-RAD) that integrates questioning with response processing and deception detection. An exploratory pilot study (N=68) and a primary study (N=225) were executed.The K-RAD was designed to have a three-dimensional figure, an "Embodied Conversational Agent" (ECA), deliver the questions through speech. This delivery mechanism was chosen because human subjects have been shown in the past to react emotionally to ECAs during conversational interactions, and emotional arousal is one of the cues to deception. Responses were analyzed for deception cues, focusing on kinesic, linguistic, and vocalic characteristics that can be captured for automated processing and which would be unique to this setting.The results show unique subject behaviors. Subjects exhibited minimal movement and had very little tendency to change posture. Some subjects (6%) referred to the ECA as an authority figure, using "sir" when responding. Subjects positioned themselves at varying distances from the ECA, with significant gender differences. Post-experiment surveys indicated a gender difference in overall stress, with female subjects reporting significantly higher levels, independent of the experimental condition.Postural-based logistic regression created significant classification models for the pilot (59.1% classification accuracy) and primary (57.2% & 62.8% classification accuracies) studies. Movement analysis had varying and conflicting results. A robust deception index with a 68.1% classification accuracy was achievable in the pilot study based on high-frequency movement and arm placement. Primary study deception indices were not significant.The results include a comprehensive set of observations and lessons learned regarding kiosk design, deception technologies, detection effectiveness, and future considerations to take into account when creating a next-generation K-RAD system. Many challenges remain, but the concept is functional, promising, and could revolutionize security screening and deception detection in a variety of settings.
Evaluating Knowledge to Support Climate Action: A Framework for Sustained Assessment. Report of an Independent Advisory Committee on Applied Climate AssessmentMoss, R. H.; Avery, S.; Baja, K.; Burkett, M.; Chischilly, A. M.; Dell, J.; Fleming, P. A.; Geil, K.; Jacobs, K.; Jones, A.; et al. (AMER METEOROLOGICAL SOC, 2019-07)As states, cities, tribes, and private interests cope with climate damages and seek to increase preparedness and resilience, they will need to navigate myriad choices and options available to them. Making these choices in ways that identify pathways for climate action that support their development objectives will require constructive public dialogue, community participation, and flexible and ongoing access to science- and experience-based knowledge. In 2016, a Federal Advisory Committee (FAC) was convened to recommend how to conduct a sustained National Climate Assessment (NCA) to increase the relevance and usability of assessments for informing action. The FAC was disbanded in 2017, but members and additional experts reconvened to complete the report that is presented here. A key recommendation is establishing a new nonfederal "climate assessment consortium" to increase the role of state/local/tribal government and civil society in assessments. The expanded process would 1) focus on applied problems faced by practitioners, 2) organize sustained partnerships for collaborative learning across similar projects and case studies to identify effective tested practices, and 3) assess and improve knowledge-based methods for project implementation. Specific recommendations include evaluating climate models and data using user-defined metrics; improving benefit-cost assessment and supporting decision-making under uncertainty; and accelerating application of tools and methods such as citizen science, artificial intelligence, indicators, and geospatial analysis. The recommendations are the result of broad consultation and present an ambitious agenda for federal agencies, state/local/tribal jurisdictions, universities and the research sector, professional associations, nongovernmental and community-based organizations, and private-sector firms.
Evaluating an APPE Assessment Tool Using Electronic Surveys to Assess Preceptor Attitudes and Implementing Improvements to More Accurately Measure Student AchievementCooley, Janet; Thoi, Sandi; Ardebili, Ranna; Boles, Hilary; Grear, Amanda; Cooley, Janet; Thoi, Sandi; College of Pharmacy, The University of Arizona (The University of Arizona., 2016)Objectives: (1) To assess preceptors’ attitudes regarding the usefulness of the web-based assessments in evaluating rotation students. (2) To determine which assessment tool (original versus modified) provides more in-depth feedback for the evaluation of students. Subjects: Preceptors for 4th year students completing their Advanced Pharmacy Practice Experiences (APPE) in a variety of pharmacy settings. Methods: Electronic surveys anonymously collected ratings of preceptor attitudes toward the original and modified APPE assessment tools. Data on the usefulness, strengths, and limitations of both tools were analyzed through thematic analysis. Results: Surveys assessing preceptor attitudes towards the original assessment tool (46 responses) and modified assessment tool (29 responses) were collected and analyzed. Similar representation was seen across all rotation settings, with an average of 7 years of precepting experience. Preceptor attitudes were more positive towards the modified tool. More preceptors “strongly agreed” or “agreed” that they were able to effectively evaluate students with the modified tool compared to the original tool (83% vs. 63%). Additionally, more preceptors “strongly agreed” or “agreed” that the modified tool incorporated all necessary competencies (79% vs. 48%) and aided student growth by addressing deficits compared to the original tool (73% vs. 50%). Conclusions: APPE preceptors perceived both the original and modified assessment tools as effective, favoring their brevity and ease of use. However, preceptor attitudes were more positive towards the modified tool. The methods utilized in this study can be implemented again for future updates of the APPE assessment tool.