Improving the interpretation and use of student ratings: A training approach

Abstract

This dissertation examines the faculty evaluation systems within the institutions of higher education today; with particular regard to how student ratings of instruction data are utilized, and by whom. Specifically, the knowledge, skills and attitudes of users of student ratings of instruction data and the effects of training on those who utilize such data. In this quasi-experimental study, a revised form of the "Using Student Ratings Data" questionnaire (Franklin & Theall, 1989) was used. The questionnaire, in its current form, consists of 70 items embedded in three subscales: knowledge, attitudes and skills; including seven demographic items. The questionnaire was designed to elicit what student ratings users knew about student ratings, their attitudes toward student ratings, and moreover, how their knowledge and attitudes affected their perceived skills in use. The revised form of the questionnaire was administered, pre and post, to participating faculty and administrators of institutes of higher learning from throughout the Continental United States and Puerto Rico. The participants were attendees at a workshop aimed at the development of a comprehensive faculty evaluation system. Obtained subscale reliability estimates of the questionnaire were moderate to high (.60-.81). Moreover, group differences were significant in both the knowledge and attitudinal subscales, suggesting that both the participant's knowledge and attitudes were positively affected by the training.