Arthritis & Rheumatism, Volume 62,
November 2010 Abstract Supplement
Abstracts of the American College of
Rheumatology/Association of Rheumatology Health Professionals
Annual Scientific Meeting
Atlanta, Georgia November 6-11, 2010.
Demonstrating Musculoskeletal Ultrasound Competency in a Valid and Reliable Way.
Kissin4, Eugene Y., Niu5, Jingbo, Thiele11, Ralf G., Nishio1, Midori Jane, Higgs13, Jay B., Goyal10, Janak R., Malone12, Daniel G.
University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, Perth Amboy, NJ
University of Rochester, Rochester, NY
University of Wisconsin, Madison, WI
Wilford Hall Medical Center/Brooke Army Medical Center, San Antonio, TX
Arthritis Rheumatic & Back Disease Association, Philadelphia, PA
Boston University, Boston, MA
Boston University, Boston, MA
Institutu Nacional de Rehab, Mexico City, Mexico
Medical Center for Rheumatology Berlin-Buch, Berlin, Germany
National Institute of Rheumatology and Physiotherapy, Budapest, Hungary
University of Florida College of Medicine, Ponte Vedra Beach, FL
The amount of training needed to become competent in musculoskeletal ultrasound (MSUS) is debatable, and a valid exam to determine competency in MSUS is needed. While others have previously established training and testing procedures, no one has yet documented MSUS test reliability or validity. We aim to develop a reliable and valid MSUS test.
USA rheumatology fellows who had participated in an 8 month, standardized, MSUS curriculum completed an exam of MSUS consisting of multiple choice questions (MCQs), and practical scanning (PrE) of normal anatomy.
10 faculty members (> 4 years of practical and teaching experience in MSUS courses) individually submitted 250 MCQs covering 8 peripheral joint regions and typical rheumatic conditions. As a group, the faculty excluded 143 MCQs that were unclear or irrelevant to rheumatology. Remaining MCQs were tested in 2 groups: 3 rheumatologists without MSUS training, to exclude MCQs answered correctly by all 3, and 5 rheumatologists expert in MSUS to exclude MCQs answered incorrectly by 3 of 5, resulting in 76 MCQs on the exam.
To validate the MCQ exam, 3 additional groups completed it: rheumatologists who finished an intermediate course of training through the Mexican School of Musculoskeletal Ultrasound (Intermediate Group (IG), n=7), rheumatologists with > 2 years of experience in MSUS from both Mexico and the USA (Advanced Group (AG), n=7), and USA fellows (Fellow Group (FG), n=19).
All Fellows and 2 MSUS faculty members took an 8-station PrE for scanning technique in all 8 joint areas. Faculty graded images from the PrE, blinded to source, on a 5 point scale based on predefined quality measures.
The MCQ exam was reliable (Cronbach's a= 0.76) and discriminated between IG and AG (median score 63% vs. 79%, p=0.003 Wilcoxon). The FG median MCQ test score exceeded that of the IG (72% vs. 63%, p=0.02 Wilcoxon), but had the greatest range - from worse than the IG to better than the AG (Figure 1). Fellows and faculty median PrE scores were not significantly different (2.4 vs. 2.8, p=0.18), but 2 fellows scored significantly lower than the faculty members. PrE and MCQ results correlated well (Spearman r=0.63, p=0.004).
A written and practical exam proved moderately reliable and valid in a comparison of practitioners of varying levels of training. The correlation between results of the two exam parts adds further validity. Most fellows who completed an 8 month MSUS training program achieved similar essential MSUS knowledge to the advanced MSUS group (MCQ), and similar basic scanning technique to their mentors (PrE), thus demonstrating basic competence. However, fellow test results varied significantly despite completion of a standardized teaching program. Given the variability in achieving MSUS competency after uniform training, requisite MSUS experience should be based on outcome testing rather than a fixed time period of education.
To cite this abstract, please use the following information:
Kissin, Eugene Y., Niu, Jingbo, Thiele, Ralf G., Nishio, Midori Jane, Higgs, Jay B., Goyal, Janak R., et al; Demonstrating Musculoskeletal Ultrasound Competency in a Valid and Reliable Way. [abstract]. Arthritis Rheum 2010;62 Suppl 10 :1617