Cranial ultrasound performance assessment tool: development and validation

Abstract

Aim: A standard assessment tool for direct evaluation of procedural skills to ensure proficiency of trainees is necessary for cranial ultrasound (US) in clinical practice. This study created and validated an assessment tool for cranial US performance by radiologists.

Material and methods: An initial evaluation tool for cranial US using criteria was developed based on existing literature. The assessment form was modified using a three-round Delphi process by an expert panel, conducted between January 2021 and April 2021. Rubric scales for grading were added once consensus regarding generated items was reached. Experts confirmed the final assessment tool using a rubric scale. Two raters evaluated cranial US performance of 27 residents in video clips using the tool. Reliability and percent agreement were assessed.

Results: Seventeen pediatric radiologists working in different settings participated in the expert panel. The content validation of the proposed evaluation tool was enabled by expert pediatric radiologists. Following three rounds of the Delphi process, the initial 14-item assessment form became a final 15-item form. A three-part rubric scale was used in the final form (preparation, US machine operation, and cranial US performance). Interrater reliability was evaluated with Cohen’s Kappa. The Kappa value and percent interrater agreement for most items was moderate to almost perfect (0.42–0.93 and 77.8–100%, respectively). The Cronbach’s alpha values for both raters were 0.856 and 0.891.

Conclusions: This study produced the first validated cranial US assessment tool using a modified Delphi method. The final assessment form is a simple and reliable tool.