fetching data ...

POS0341 (2023)
PERFORMANCE ANALYSIS OF A DEEP LEARNING ALGORITHM TO DETECT POSITIVE SIJ MRI ACCORDING TO THE ASAS DEFINITION IN AXSPA PATIENTS
J. Nicolaes1,2, E. Tselenti3, T. Aouad4, C. López-Medina5,6,7, A. Feydy6,8, H. Talbot4, B. Hoepken9, N. De Peyrecave2, M. Dougados5,6
1KU Leuven, Department of Electrical Engineering (ESAT), Center for Processing Speech and Images, Leuven, Belgium
2UCB Pharma, N/A, Brussels, Belgium
3Veramed, N/A, London, United Kingdom
4Universite Paris-Saclay, CentraleSup´elec, Inria. Gif-sur-Yvette, France
5Cochin Hospital, Rheumatology Department, Paris, France
6University of Paris, INSERM (U1153): Clinical Epidemiology and Biostatistics, Paris, France
7Reina Sofia Hospital, Cordoba/ IMIBIC/ University of Cordoba, Rheumatology Department, Cordoba, Spain
8Cochin Hospital, Radiology Department, Paris, France
9UCB Pharma, N/A, Monheim am Rhein, Germany

 

Background Magnetic resonance imaging (MRI) of the sacroiliac joints (SIJ) is an essential tool in the evaluation of patients with axial spondyloarthritis (axSpA). In-depth knowledge of characteristic MRI lesions and their definitions, as well as reliability of identification and scoring, varies amongst general radiologists and rheumatologists.[1] A deep learning algorithm was developed to detect the presence of inflammation in SIJ MRI (MRI+) scans with promising results.[2]

Objectives The aim of this diagnostic performance study was to assess the ability of a deep learning algorithm to identify MRI+ scans in a study cohort of axSpA patients.

Methods 731 baseline SIJ MRI scans were collected from two prospective randomised controlled trial cohorts in patients with non-radiographic (nr-) and radiographic (r-) axSpA (RAPID-axSpA [NCT01087762] and C-OPTIMISE [NCT02505542])[3,4] and were centrally evaluated by two expert readers (and adjudicator in case of disagreement) for the presence of inflammation by the 2009 Assessment in SpondyloArthritis international Society (ASAS) definition.[5] The MRI scans were processed by the previously trained deep learning algorithm,[2] blinded to clinical information and central expert readings.

Performance evaluation included sensitivity, specificity, positive and negative predictive values (PPV and NPV), Cohen’s Kappa and the absolute agreement to assess the agreement between the deep learning algorithm and the human readers for the classification of MRI-SIJ scans. Bootstrapping was used to construct the 95% confidence interval (CI).

Results Pooling the patients from RAPID-axSpA (n=152) and C-OPTIMISE (n=579) yielded a validation set of 731 patients (mean age: 34.2 years, SD: 8.6; 69.1% male) of which 44.6% were patients with nr-axSpA and 59.6% were MRI+ as per central readings.

Comparing the trained algorithm with the human central readings for the classification of MRI+/MRI– on the pooled validation set yielded a sensitivity of 70% (95% CI: 66–73%), specificity of 81% (95% CI: 78–84%), PPV of 84% (95% CI: 82–87%), NPV of 64% (95% CI: 61–68%), Cohen’s kappa of 0.49 (95% CI: 0.43–0.55), and absolute agreement of 74% (95% CI: 72–77%; Table 1).

Conclusion A previously trained deep learning algorithm enabled acceptable detection of the presence of inflammation according to the 2009 ASAS MRI definition in axSpA patients from two clinical trials. This suggests that an MRI+ detection algorithm has the potential to support clinicians in identifying axSpA patients.

References

  1. Bennett AN. J Rheumatol 2017;44(6):780–5;
  2. Aouad T. Proc Int Conf Image Proc 2022;3351–5;
  3. van der Heijde. Rheumatology. 2017;56(9):1498–1509;
  4. Landewé RB. Ann Rheum Dis. 2020;79(7):920–28;
  5. Rudwaleit M. Ann Rheum Dis 2009;68(6):777–83.

Table 1. Performance results comparing the algorithm and the human readers for the classification of MRI-SIJ scans. The metric values are point estimate (95% CI).
Metric All (N=731) RAPID-axSpA (N=152) C-OPTIMISE (N=579)
Central reading, MRI+; n(%) 436 (59.6%) 99 (65.1%) 337 (58.2%)
Sensitivity 0.70(95% CI: 0.66–0.73) 0.66(95% CI: 0.58–0.73) 0.71(95% CI: 0.67–0.75)
Specificity 0.81(95% CI: 0.78–0.84) 0.89(95% CI: 0.82–0.95) 0.79(95% CI: 0.75–0.83)
PPV 0.84(95% CI: 0.82–0.87) 0.92(95% CI: 0.87–0.96) 0.83(95% CI: 0.79–0.86)
NPV 0.64(95% CI: 0.61–0.68) 0.58(95% CI: 0.50–0.67) 0.66(95% CI: 0.62–0.70)
Cohen’s kappa 0.49(95% CI: 0.43–0.55) 0.48(95% CI: 0.36–0.61) 0.49(95% CI: 0.42–0.56)
Absolute agreement 0.74(95% CI: 0.72–0.77) 0.74(95% CI: 0.68–0.79) 0.74(95% CI: 0.72–0.77)

CI: confidence interval; MRI: magnetic resonance imaging; NPV: negative predictive value; PPV: positive predictive value; SIJ: sacroiliac joints.

Acknowledgements We thank the patients who participated. Funded by UCB Pharma. Editorial support provided by Costello Medical and funded by UCB Pharma.

Disclosure of Interests Joeri Nicolaes Shareholder of: UCB Pharma, Employee of: UCB Pharma, Evi Tselenti Employee of: Veramed statistical consultant for UCB Pharma, Theodore Aouad: None declared, Clementina López-Medina Speakers bureau: Eli Lilly, Novartis, UCB Pharma, MSD, Abbvie and Janssen, Consultant of: UCB Pharma, Eli-Lilly, and Novartis, Grant/research support from: Abbvie, UCB Pharma, Eli Lilly, and Novartis, Antoine Feydy Consultant of: Guerbet, Hugues Talbot: None declared, Bengt Hoepken Shareholder of: UCB Pharma, Employee of: UCB Pharma, Natasha de Peyrecave Employee of: UCB Pharma, Maxime Dougados Speakers bureau: UCB Pharma, Grant/research support from: UCB Pharma.

Keywords: Spondyloarthritis, Artificial Intelligence, Imaging

DOI: 10.1136/annrheumdis-2023-eular.2357

Citation: , volume 82, supplement 1, year 2023, page 418
Session: Beyond the crystal ball (Poster Tours)