Evaluating the reliability and validity of a questionnaire used to measure experiences of teamwork among student pharmacists in a quality improvement course

Abstract

Introduction: The psychometric properties of instruments used to capture student pharmacists' perspectives of teamwork have not been well assessed. This study measured the reliability and validity of an instrument designed to assess teamwork experiences among student pharmacists in a quality improvement (QI) class at one United States pharmacy school. Methods: The psychometric properties of a previously conducted 17-item questionnaire (response options: “strongly agree,” “agree,” “disagree,” or “strongly disagree”) about second-year student pharmacists' teamworking experiences were assessed. A Rasch rating scale model was used to construct measures of teamwork experience. Principal component analysis (PCA) assessed unidimensionality. Item- and person-fit statistics were assessed. Construct and content validity and reliability were estimated utilizing student and item separation indices (SI) and reliability coefficients (RC). Results: Sixty student pharmacists were included. PCA conveyed a unidimensional construct. Four items with infit and outfit mean-squared values outside the suggested range were removed. Item responses “disagree” and “strongly disagree” were merged to improve scale functionality. The average person measure was 1.74 ± 2.03 logits. Student and item RC were 0.81 (SI = 2.04) and 0.97 (SI = 2.17), respectively. The easiest item endorsed was team's ability to reach consensus, while the most difficult item was interest to do collaborative work again. Mismatch of student experience and item difficulty level on the continuum scale suggested additional items are needed to match student teamwork experience. Conclusion: The instrument demonstrated evidence of reliability and validity to measure student pharmacists' teamwork experience in a QI class, but additional instrument modifications are recommended.