Background: Musculoskeletal ultrasound (MSUS) has established itself as a disease outcome measurement instrument over the last two decades and is now widely used in clinical practice and research. MSUS target domains, including consensual theoretical definitions of pathological processes (e.g. synovitis or structural damage), operational definitions of elementary lesions (e.g. synovial hypertrophy or bone erosions), acquisition standards and scoring systems developed as part of the Outcome Measures in Rheumatology (OMERACT) methodology, have been validated via a step-wise approach consisting of Delphi process to produce theoretical and operational definitions, and intra- and interobserver reliability exercises based either on still images or videos, or on live acquisition using patients or healthy subjects to test such theoretical agreement.

Objectives: Evaluate factors influencing Delphi- and reliability exercises in MSUS for developing recommendations to standardize consensus and validation studies aiming at producing new scoring systems.

Methods: A systematic search of the Central, Embase, and Medline databases was performed by two independent reviewers in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Original articles published in English up to September 2023, reporting on Delphi consensus studies and intra- and interobserver reliability exercises performed on adult patients or healthy subjects according to the OMERACT Ultrasound Working Group (US WG) methodology were included. Attributes of the consensus studies and reliability exercises were extracted from the included studies using a predefined sheet.

Results: A total of 2744 articles were screened, and 25 were included in the final review. Of these, 15 reported on Delphi studies, while 25 reported on 33 reliability exercises (16 image-based and 17 patient-based). The majority of the Delphi studies included the target domains “elementary lesions” (87%) and scoring systems (73%) (Table 1). 86% of these studies consisted of up to 3 rounds, and 86% utilized ≥75% as a cut-off for reaching agreement among participants. The number of participants ranged from 13 to 49, with response rates between 68-100%. With respect to reliability exercises, the number of images used ranged from 23 to 152, the number of patients from 6 to 13 and the number of expert participants from 12-35 (image-based) and 9-19 (patient-based) (Figure 1). All but one exercise featured two rounds, and all of them assessed both intra- and interobserver reliability. A single exercise used various forms of kappa statistics, while another single exercise utilised an intraclass correlation coefficient to measure reliability. All but a single (97%) of reliability studies evaluated either elementary lesions and/or scoring systems. While 59% of patient-based reliability exercises featured a pre-training exercise, this was the case only in 6% of image-based exercises.

Conclusion: The included Delphi consensus and reliability exercise studies focused on either elementary lesions or scoring systems. Variation was observed in the number of included expert participants, of images/patients utilized, and the integration of pre-training exercises. These results represent the first step by the US WG for producing recommendations to harmonise consensus and reliability studies in MSUS, the results of which will aid OMERACT groups and other researchers to develop and validate new definitions or scoring systems for imaging outcome measures.

REFERENCES: NIL.

Table 1. Key attributes of Delphi consensus studies in musculoskeletal ultrasound according to the OMERACT Ultrasound Working Group methodology. AC: acquisition standard, EL: elementary lesion; GD: global definition; NA: not available; SS: scoring system

Figure 1. Key attributes of intra-/interobserver reliability exercises in musculoskeletal ultrasound according to the OMERACT Ultrasound Working Group methodology. EL: elementary lesion; GD: global definition; NA: not available; SS: scoring system

Acknowledgements: NIL.

Disclosure of Interests: Peter Mandl AbbVie, Janssen, AbbVie, Novartis, Janssen, Stine Maya Dreier Carstensen: None declared, Lene Terslev Janssen, Novartis and GE, Helen Keen: None declared, Carlos Pineda: None declared, Maria-Antonietta D’Agostino: None declared.