Background: Hand motion tracking offers significant potential for remote assessment of disease activity in rheumatoid arthritis (RA) patients, enabling telemedicine and improving access to care [1]. Current systems rely on specialized hardware, limiting their applicability in remote or resource-limited settings. Advances in deep learning and computer vision provide opportunities to simplify hand motion tracking using widely available devices like smartphones. Telemedicine, especially in rheumatology, can enhance workflow efficiency, improve access for disabled or remote patients, and reduce costs compared to in-person visits. Given the chronic nature of rheumatic diseases, objective tools for remote disease monitoring are increasingly necessary.

Objectives: This study aims to explore the association between hand motion tracking features, derived via computer vision from smartphone cameras, and disease activity in RA patients.

Methods: A multicenter study was conducted involving RA patients from three European rheumatology centers (Bari, Lausanne, Bern). Disease activity was measured using the DAS28-ESR, Swollen Joint Count (SJC), Tender Joint Count (TJC), and Health Assessment Questionnaire-Disability Index (HAQ-DI). An ad-hoc app leveraging the MediaPipe API was developed in Python and deployed on Windows 11 for hand motion analysis. Participants performed five rapid finger flexion iterations with their dominant hand while being recorded using a smartphone camera. The algorithm quantified joint angle changes (proximal interphalangeal -PIP- distal interphalangeal -DIP- and metacarpophalangeal -MCP- joints) and time to maximal flexion [Figure 1A]. Patients with established hand deformities (e.g., swan neck, boutonniere, ulnar deviations) were excluded. Kinetic variables were analyzed using multivariate logistic regression models to predict low disease activity/remission (DAS28). Correlations with HAQ-DI were assessed using Spearman’s ρ. Prediction performance was validated using 5-fold cross-validation (n=81) and an external dataset (n=19) [Figure 1B].

Results: The model demonstrated robust performance in predicting low disease activity/remission. Internal validation yielded a mean accuracy of 0.72 ± 0.09, specificity of 0.65 ± 0.17, recall of 0.86 ± 0.05, and AUROC of 0.80 ± 0.09. External validation showed improved performance, with an accuracy of 0.84, specificity of 0.75, recall of 0.91, and AUROC of 0.89 [Figure 1B]. Greater joint angle changes and shorter time to maximal flexion correlated with lower disease activity. Significant associations were observed between kinetic features and clinical measures, including DAS28, SJC, TJC, and HAQ-DI [Figure 1C].

Conclusion: Computer vision-based hand motion tracking can effectively distinguish RA disease activity states and correlates with clinical and patient-reported outcome measures. This technology offers a promising avenue for telemedicine and remote monitoring, complementing subjective assessments with objective metrics. Future research should investigate potential confounding factors, such as hand deformities or osteoarthritis, and explore the integration of hand motion tracking into comprehensive remote monitoring platforms for RA patients.

REFERENCES: [1] Salchow-Hömmen C, Callies L, Laidig D, Valtin M, Schauer T, Seel T. A Tangible Solution for Hand Motion Tracking in Clinical Applications. Sensors (Basel). 2019 Jan 8;19(1):208.

Acknowledgements: NIL.

Disclosure of Interest: None declared.

© The Authors 2025. This abstract is an open access article published in Annals of Rheumatic Diseases under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ ). Neither EULAR nor the publisher make any representation as to the accuracy of the content. The authors are solely responsible for the content in their abstract including accuracy of the facts, statements, results, conclusion, citing resources etc.