Arthritis & Rheumatism, Volume 62,
November 2010 Abstract Supplement
Abstracts of the American College of
Rheumatology/Association of Rheumatology Health Professionals
Annual Scientific Meeting
Atlanta, Georgia November 6-11, 2010.
The First Pediatric Rheumatology Objective Structured Clinical Examination: Providing Clinical Skills Feedback for Fellows and Program Directors.
Curran1, Megan L., Sherry2, David D.
Since objective data is often lacking, it is difficult for fellowship program directors (PDs) to provide meaningful feedback regarding fellows' communication and professionalism skills. To reliably assess these non-knowledge based competencies we conducted an OSCE for pediatric rheumatology fellows to measure pre-test expectations and actual scores in communication and professionalism skills and overall performance rank.
22 fellows from 13 training programs were tested in 7 unique scenarios. The 21 evaluators were pediatric rheumatologists mostly from participating programs. Before the OSCE, fellows and PDs used 100 mm visual analog scales (VAS) to indicate expected performance in communication and professionalism. Evaluators scored fellows using identical VAS during scenarios. Scenario scores were averaged to give final scores in these areas. Before the OSCE, fellows and PDs estimated overall performance (upper, middle or lower third) compared to others. After all 7 fellows finished an evaluator's station, their performances were ranked from 1 (best) to 7 (worst). Rankings were averaged to determine each fellow's overall performance score. Scores were ordered numerically and divided into upper, middle and lower thirds.
The 22 fellows' pre-test communication VAS estimates ranged 2280 (56, 57, and 49 were the mean, median and mode). Post-test communication VAS scores ranged 2689 (78, 80, 86). Fellows' pre-test professionalism VAS estimates ranged 3998 (60, 60, 50). Post-test professionalism VAS scores ranged 2690 (84, 88, 88). Upper, middle and lower third overall performance rank averages ranged respectively from 1.25 to 3.33 (n=8), 3.43 to 4.29 (n=7) and 4.43 to 5.83 (n=7). The following statistics show the number of fellows who estimated their performance ranking to fall into the specified third with the number actually achieving upper, middle, and lower third rankings listed in parentheses. 2 fellows estimated their performance as upper third (1, 0, 1), 12 as middle third (6, 4, 2) and 8 as lower third (1, 3, 4). 1st year fellows estimated their performance ranking to be lower than 2nd year fellows, whose estimates were lower than estimates of 3rd year fellows. PDs estimated 10 of their fellows' performances as upper third (4, 3, 3) and 4 as middle third (1, 2 and 1). No program director thought that their fellow would rank in the lower third. With few exceptions, attendings expected 1st year fellows to rank in the middle third and 2nd & 3rd year fellows to rank in the top third.
Our pediatric rheumatology OSCE was the first of its kind. Fellows tended to underestimate their communication skills, professionalism skills and overall performance ranking. PDs overestimated performances. Limitations of our study include inter-rater reliability of evaluators and comparison of fellows from different training levels. However, the pediatric rheumatology OSCE provided innovative feedback for program directors about their fellows' communication and professionalism skills. This OSCE provides the first opportunity to compare the clinical skills of pediatric rheumatology fellows from programs nationwide.
To cite this abstract, please use the following information:
Curran, Megan L., Sherry, David D.; The First Pediatric Rheumatology Objective Structured Clinical Examination: Providing Clinical Skills Feedback for Fellows and Program Directors. [abstract]. Arthritis Rheum 2010;62 Suppl 10 :47