Milestone Ratings and Supervisory Role Categorizations Swim Together, but Is the Water Muddy?
AuthorSchumacher, Daniel J
Bartlett, Kathleen W
Elliott, Sean P
Garfunkel, Lynn C
AffiliationUniv Arizona, Dept Pediat
Univ Arizona, Phoenix Childrens Hosp, Dept Pediat
Keywordsclinical competency committee
graduate medical education
MetadataShow full item record
PublisherELSEVIER SCIENCE INC
CitationSchumacher, D. J., Bartlett, K. W., Elliott, S. P., Michelson, C., Sharma, T., Garfunkel, L. C., ... & APPD LEARN CCC Study Group. (2019). Milestone Ratings and Supervisory Role Categorizations Swim Together, but Is the Water Muddy?. Academic Pediatrics, 19(2), 144-151.
RightsCopyright © 2018 by Academic Pediatric Association
Collection InformationThis item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at email@example.com.
AbstractThis single-specialty, multi-institutional study aimed to determine 1) the association between milestone ratings for individual competencies and average milestone ratings (AMRs) and 2) the association between AMRs and recommended supervisory role categorizations made by individual clinical competency committee (CCC) members. METHODS: During the 2015-16 academic year, CCC members at 14 pediatric residencies reported milestone ratings for 21 competencies and recommended supervisory role categories (may not supervise, may supervise in some settings, may supervise in all settings) for residents they reviewed. An exploratory factor analysis of competencies was conducted. The associations among individual competencies, the AMR, and supervisory role categorizations were determined by computing bivariate correlations. The relationship between AMRs and recommended supervisory role categorizations was examined using an ordinal mixed logistic regression model. RESULTS: Of the 155 CCC members, 68 completed both milestone assignments and supervision categorizations for 451 resi- dents. Factor analysis of individual competencies controlling for clustering of residents in raters and sites resulted in a single -factor solution (cumulative variance: 0.75). All individual competencies had large positive correlations with the AMR (correlation coefficient: 0.84-0.93), except for two professionalism competencies (Profl : 0.63 and Prof4: 0.65). When combined across training year and time points, the AMR and supervisory role categorization had a moderately positive correlation (0.56). CONCLUSIONS: This exploratory study identified a modest correlation between average milestone ratings and supervisory role categorization. Convergence of competencies on a single factor deserves further exploration, with possible rater effects warranting attention.
Note12 month embargo; published online: 18 June 2018
VersionFinal accepted manuscript
- Influence of Clinical Competency Committee Review Process on Summative Resident Assessment Decisions.
- Authors: Schumacher DJ, King B, Barnes MM, Elliott SP, Gibbs K, McGreevy JF, Del Rey JG, Sharma T, Michelson C, Schwartz A, Members of the APPD LEARN CCC Study Group.
- Issue date: 2018 Aug
- Justifications for Discrepancies Between Competency Committee and Program Director Recommended Resident Supervisory Roles.
- Authors: Schumacher DJ, Poynter S, Burman N, Elliott SP, Barnes M, Gellin C, Gonzalez Del Rey J, Sklansky D, Thoreson L, King B, Schwartz A, APPD LEARN CCC Study Group.
- Issue date: 2019 Jul
- Competent for Unsupervised Practice: Use of Pediatric Residency Training Milestones to Assess Readiness.
- Authors: Li ST, Tancredi DJ, Schwartz A, Guillot AP, Burke AE, Trimm RF, Guralnick S, Mahan JD, Gifford KA, Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN) Validity of Resident Self-Assessment Group.
- Issue date: 2017 Mar
- Correlations Between Ratings on the Resident Annual Evaluation Summary and the Internal Medicine Milestones and Association With ABIM Certification Examination Scores Among US Internal Medicine Residents, 2013-2014.
- Authors: Hauer KE, Vandergrift J, Hess B, Lipner RS, Holmboe ES, Hood S, Iobst W, Hamstra SJ, McDonald FS
- Issue date: 2016 Dec 6
- A multi-source feedback tool for measuring a subset of Pediatrics Milestones.
- Authors: Schwartz A, Margolis MJ, Multerer S, Haftel HM, Schumacher DJ, APPD LEARN–NBME Pediatrics Milestones Assessment Group.
- Issue date: 2016 Oct