An Analysis of Differences in Non-Instructional Factors Affecting Teacher-Course Evaluations over Time and Across Disciplines

Abstract

This dissertation looked at the relationship between students' evaluations of teaching (SET) at a large research university in the United States and a set of background variables comprised of nine course, instructor, and student characteristics. Data from over 130,000 course evaluations from over 4,000 courses from four distinct departments taught between 2007 and 2014 were analyzed. Student ratings have been used to formally evaluate effective teaching practices at all levels of education for nearly 100 years. The subsequent body of literature examining and challenging this practice is vast and continuously evolving, and largely built on issues of validity, reliability, and bias. The findings have varied considerably over the years, largely due to the institutional-uniqueness of the instruments being used, the differing methodologies used to analyze the data, and disagreement on how to interpret the findings. These issues have allowed SET to continue to be one of the most widely studied and debated topics found in the educational literature. Findings from this study provide further evidence that SET data should not be used to make broad comparative judgments, but are more appropriate as a measure to inform individual instructors. Significant differences were detected from all nine background variables, with meaningful differences observed at the departmental level. While some of the variance in ratings detected can be logically tied to evidence of effective teaching practices, others indicate potential unfair biases that could be harmful if precautions are not taken in how the data are distributed and used.