Background: Assessing (teno-)synovitis and bone marrow edema (BME) in the wrist of patients with (suspected) rheumatoid arthritis from magnetic resonance imaging (MRI) is generally done by visual scoring. Since this is a time-consuming task needing extensive training, we propose an automatic method based on Deep Learning (DL).

Objectives: To automatically assess wrist MRI by deep learning, we defined two objectives. First, bones, tendons and remaining anatomical structures need to be detected (‘segmentation’). Subsequently, inflammation needs to be measured around each tendon (group) for quantifying tenosynovitis, in the synovial areas for quantifying synovitis and within the bones for measuring BME, while carefully excluding remaining anatomical structures (like large vessels and skin) from these measurement areas.

Methods: A convolutional neural network (U-net) was combined with post-processing. Post-contrast axial and coronal MRI scans of the wrist of 1225 consecutively identified early arthritis patients were used for training and optimization. From 28 patients, manual segmentations were used as atlases, containing 14 flexor/extensor tendon groups; 10 carpal, 5 metacarpal bones; 1 label for all main vessels; 2 labels for skin and remaining tissue; and 1 background label. Data were normalized with lower and upper thresholds (0.01, 0.99) and augmented by intensity scaling (0.8-1.25) and adding Gaussian noise (µ= 0, σ=0.06). After 1-fold cross-validation, we trained two networks for segmenting coronal and axial images with all atlases. After segmentation, measurement areas were determined, following the RAMRIS definition: starting from the location where radius and ulna are closest to each other, ending with the hook of the hamate. We used the 25 ^th percentile of the histogram from the blood vessels to define calibrated inflamed regions within the measurement areas. In an area 3 mm around each tendon (excluding bones and vessels) tenosynovitis was quantified by the ratio of the number of ‘inflamed pixels’ to the total area. Similarly, synovitis was measured from the distal radioulnar, radiocarpal, intercarpal and carpometacarpal joints, and BME was measured inside the carpals bones. We checked the segmentation-training by plotting the overall Dice accuracy (the relative overlap with the true segmentation) against epoch number. The quantification was validated by computing the with Pearson correlations coefficients between automated and manual scoring (the mean from two readers).

Results: An example of an automatic segmentation result is shown in Figure 1a. The training curves showed a rapid convergence, especially in the coronal images (Figure 1b). The quantifications correlated with the visual scores, computed for 1225 patients: r = 0.87 for tenosynovitis, r = 0.83 for synovitis, r = 0.59 for BME, p < 0.001, respectively (Figure 2). Because of the small BME range and segmentation errors, the correlation in BME was relatively low.

Conclusion: Our deep learning model provides a first step into automatically quantifying joint inflammation from MRI of the wrist. Whilst it is much faster (~10 sec per patient) than visual scoring, it performs well for tenosynovitis and synovitis but less so for BME. This approach is therefore dependent on (bone) segmentation accuracy. We presented the total measurements of all (teno-)synovial and bone areas, but automated quantification per tendon/area or bone is also available. These results provide a basis for automatic quantifications, especially for tenosynovitis and synovitis.

Figure 1.

a) Example of an axial and coronal MRI image with segmentation overlays. b) Plot of the wrist segmentation accuracy during training, according to the Dice coefficient, separately for the axial and coronal MRI scans.

Figure 2.

Left column: scatter plots of total quantitative measurements of (teno-)synovitis (Tq and Sq) and BME (Bq) versus total visual scores for 1225 patients (Tμ, Sμ and Bμ). The dashed lines represent linear regression fit. Right column: examples of scoring areas (in blue) and detected inflamed areas (in pink).

REFERENCES: NIL.

Acknowledgements: This AI-project has been funded by the Dutch Research Council (NWO) Applied and Engineering Sciences (project number 17970). The TREAT-EARLIER trial has been funded by an NWO-ZonMW grant (project number 95104004). The Dutch Arthritis Society contributed financially to both grants.

Disclosure of Interests: Denis Shamonin Grant/research support from: Bristol-Myers Squibb and Pfizer contributed to this project, through a grant from the Dutch Research Council (NWO), Applied and Engineering Sciences., Yanli Li: None declared, Tahereh Hassanzadeh Grant/research support from: Bristol-Myers Squibb and Pfizer contributed to this project, through a grant from the Dutch Research Council (NWO), Applied and Engineering Sciences., M. Els Bakker: None declared, Dennis A. Ton: None declared, Monique Reijnierse Grant/research support from: Bristol-Myers Squibb and Pfizer contributed to this project, through a grant from the Dutch Research Council (NWO), Applied and Engineering Sciences., Annette H.M. van der Helm – van Mil Grant/research support from: Bristol-Myers Squibb and Pfizer contributed to this project, through a grant from the Dutch Research Council (NWO), Applied and Engineering Sciences., Berend Stoel Grant/research support from: Bristol-Myers Squibb and Pfizer contributed to this project, through a grant from the Dutch Research Council (NWO), Applied and Engineering Sciences.