Background: Hand motion tracking offers promising possibilities for remotely assessing disease activity in rheumatoid arthritis (RA) patients, enabling telemedicine and improving access to care. One limitation of the current systems for hand motion tracking is the reliance on dedicated camera systems and/or sensors, which may not be readily available in all clinical settings or for remote patient monitoring. Deep learning has the potential to enhance hand motion tracking significantly [1].

Objectives: This proof-of-concept study investigated the association between hand motion tracking features obtained through computer vision on smartphone cameras and disease activity in RA patients.

Methods: This proof-of-concept study included patients with classified RA from three European Rheumatology centres. Disease activity was assessed using the Erythrocyte Sedimentation Rate -Disease Activity Score 28 (DAS28-ESR), the Swollen Joint Count (SJC), and the Tender Joint Count (TJC).

Health Assessment Questionnaire-Disability Index (HAQ-DI) was also recorded.

An ad-hoc hand motion analysis app based on the MediaPipe API ( https://developers.google.com/mediapipe ) was developed in a Python 3.9 environment and deployed on Windows 11, enabling computer vision inference of hand motion data. Patients were asked to perform finger flexion, making a fist for five iterations with the dominant limb at the highest possible speed (sample video: https://tinyurl.com/4hux35ze ). Upon smartphone single camera recording, the algorithm tracked the mean degree change of joint angle on flexion in proximal and distal interphalangeal (PIP, DIP) and metacarpophalangeal (MCP) joints and the mean time to maximal flexion for five iterations in seconds. (Figure 1A). Patients with established RA hand deformities (swan neck, boutonniere, ulnar deviations) were excluded.

We tested the association of kinetic covariates with DAS28 using a multivariate logistic regression model. The prediction performance was further analyzed using the area under the receiver operating characteristic (AUROC) curve. We also investigated the correlation between kinetics variables and HAQ-DI using Spearman’s ρ.

Results: Eighty-one patients (72/81 female, 88.9%) were enrolled. The mean (±SD) age of the patients was 57.70 ± 12.83. The mean DAS28 for the patients was 3.27 ± 1.33. Among the 81 patients, 47 (58%) were classified as having low disease activity (LDA) or in remission, whereas the rest had moderate-to-high disease activity (HDA, 34/81, 41.98%). The mean degree change of either PIP (OR 1.04, 95%CI 1.02-1.07) and DIP (OR 1.03, 95%CI 1.01-1.07) joints together with mean time to maximal flexion (OR 0.18, 95%CI, 0.04-0.81) predicted DAS28-LDA remission with AUROC of 0.80 (Figure 1 B). HAQ-DI showed a moderate-to-strong correlation with mean time to maximal flexion (ρ=0.49, p=0.0007, Figure 1 C).

Conclusion: This proof-of-concept study demonstrated that joint function assessed by computer vision correlates with clinimetric indexes and patient-reported outcome measures in RA patients in a real-world setting. Further research is needed to investigate whether hand deformities or osteoarthritis may influence hand-motion tracking performance. Integrating objective hand motion tracking into remote patient monitoring platforms could complement subjective patient-reported outcomes or other digital biomarkers.

REFERENCES: [1] Salchow-Hömmen C, Callies L, Laidig D, Valtin M, Schauer T, Seel T. A Tangible Solution for Hand Motion Tracking in Clinical Applications. Sensors (Basel). 2019 Jan 8;19(1):208.

Figure 1.

Acknowledgements: NIL.

Disclosure of Interests: Vincenzo Venerito Lilly, Galapagos, Abbvie, Galapagos, Lilly, Novartis, Boehringer Ingelheim, Janssen, GSK, Galapagos, Lilly, Galapagos, Abbvie, Tobias Manigold: None declared, Marco Capodiferro: None declared, Deborah Markham: None declared, Marc Blanchard: None declared, Florenzo Iannone: None declared, Thomas Hügle: None declared.