Background: Photo documentation with subsequent automated image analysis of joints can complement patient reported outcomes (PROs) in remote patient monitoring in patients with rheumatoid arthritis (RA). We previously reported a ratio of automated joint diameter measurement and finger fold length in proximal interphalangeal (PIP) joints as a proxy for joint swelling. The accuracy and acceptance of self-imaging have not been studied so far.

Objectives: To compare the quality of hand images captured by patients using smartphones with those captured by health care professionals (HCP) for AI-based processing and suitability for finger fold analysis, and to evaluate the agreement of the Finger Fold Index (FFI), a digital biomarker of joint swelling, derived from images acquired by patients and health care professionals.

Methods: Nine rheumatology departments in Switzerland were trained to collect standardized hand photographs either via smartphone or tablet from patients with RA and psoriatic arthritis (PsA) during routine consultations using the SCQM registry. In addition, patients submitted PROs and images of both hands via the mySCQM mobile application. A photo function has been integrated into mySCQM application, which provided identical standardized instructions for image acquisition (Figure 1). Instructions included capturing an image of the dorsal side of each hand, placed flat on a white surface with the fingers slightly spread, using a vertical orientation at a distance of 20–30 cm. Patients were instructed to capture hand images on the day of their rheumatology visit or within a ±7-day window, as well as whenever PROs were submitted, typically on a monthly basis. Hand images were analyzed using a machine-learning pipeline for automated detection and processing of PIP joints. The algorithm quantified the fold surface area relative to joint diameter, yielding the FFI. FFI values derived from HCP and patient-captured images were compared using intraclass correlation coefficients (ICC) and Bland–Altman plots to assess agreement.

Results: 374 patients were included. 920 photos taken by HCPs during 467 visits. 42% (147/347) of the patients provided 1285 self-captured images via mySCQM. 13% (324/2529) of the images had to be removed due to inadequate file formats. 87% (2205/2529) of the images collected were analyzed with the machine learning pipeline to obtain the digital biomarker (FFI). No results was available for 21 visits due to incorrect hand positioning, deformation or low resolution. Only 16/446 visits, had images taken by the patient on the same day as the visit (±7 day window) with the HCP. Agreement between HCP and patient-derived FFI (N = 66) was poor, as indicated by a low ICC value of 0.51 (95% CI [-0.192─0.288]) with a large span of limits of agreement [-0.085─0.037] (Figure 2). The ICC for the average of measurements was 0.097 (95% CI [-0.475− 0.447]), also indicating poor reliability between raters.

Conclusions: Self-imaging by patients with RA and PsA showed high acceptance within the SCQM registry and may be of clinical value for remote patient monitoring. However, image quality varied despite training, indicating the need for better standardization of image capture and further investigation into the accuracy of the machine learning pipeline.

Figure 1.

Integration of the photo function in the mySCQM patient application (A) and the SCQM registry dashboard (B). The quality of self-acquired images varied, with photos taken according to the instructions considered usable (C), whereas others were non-usable (D). Higher image quality was achieved when photographs were taken by healthcare professionals (E).

Figure 2.

Bland-Altman Plot to illustrate the difference in the Finger Fold Index derived from images taken by health care professionals vs taken by patients.

REFERENCES: [1] Hügle T, Caratsch L, Caorsi M, Maglione J, Dan D, Dumusc A, Blanchard M, Kalweit G, Kalweit M. Dorsal Finger Fold Recognition by Convolutional Neural Networks for the Detection and Monitoring of Joint Swelling in Patients with Rheumatoid Arthritis. Digit Biomark. 2022 Jun 8;6(2):31-35. doi: 10.1159/000525061. PMID: 35949225; PMCID: PMC9247561.

Acknowledgments: NIL.

Disclosure of Interests: Cinja Nadana Koller Research grant: Fresenius Kabi, Jules Maglione: None declared, Marc Blanchard Atreon SA, Almut Scherer: None declared, Michael Andor: None declared, Laure Brulhart Bletsas: None declared, Raphael Micheroli: None declared, Andrea Rubbert-roth: None declared, Tobias Manigold: None declared, Christof Iking-Konert: None declared, Chrysoula Manolaraki: None declared, Alexandre Dumusc: None declared, Michael Nissen: None declared, Thomas Hügle Abbvie, GSK, J&J, UCB, Fresenius Kabi.