Background: Early and accurate diagnosis of rheumatic diseases remains a critical challenge, with musculoskeletal symptoms accounting for nearly 30% of all general practice consultations [1, 2]. While traditional anamnesis and structured interviews are fundamental tools, they may miss crucial patient-reported information, leading to diagnostic delays and potential irreversible joint damage [3, 4, 5]. Studies show that up to 80% of diagnostic accuracy can be attributed to thorough symptom history [6], yet current diagnostic support systems often fail to capture the nuances of patients’ experiences. Natural language processing (NLP) and machine learning (ML) present innovative solutions for systematically analyzing patients’ own symptom descriptions.

Objectives: To develop and validate an NLP and ML-based system for predicting diagnoses of osteoarthritis (OA), fibromyalgia (FM), and immune-mediated rheumatic diseases (imRD) from patients’ free-text symptom descriptions. Additionally, to evaluate the system’s performance in distinguishing between new presentations and previously diagnosed cases, addressing real-world clinical scenarios.

Methods: Using NLP and ML techniques, we analyzed free-text responses from participants who completed the Rheumatic® survey between June 2021 and February 2023. Participants provided symptom descriptions (maximum 200 words) at baseline and completed follow-up surveys at 3, 6, and 12 months. Support Vector Machine (SVM) models were developed using term frequency-inverse document frequency (TF-IDF) vectorization and validated through cross-validation (2x5). Models were strategically optimized for disease-specific clinical needs: maximizing positive predictive value for OA and FM, and sensitivity for imRD to minimize missed diagnoses.

Results: We analyzed free – text from 8,454 participants (78% female, 75% aged >50) with 3930, 1457, 1747 and 2393 patients having OA, FM, imRD or being healthy controls respectively. Our SVM models demonstrated robust discriminatory performance across different rheumatic diseases (Figure 1). For existing diagnoses, the models achieved area under the curve receiver operating characteristic (AUC-ROC) scores of 0.69 (95% CI 0.67-0.72) for OA, 0.76 (95% CI 0.73-0.79) for FM and 0.71 (95% CI 0.68-0.74) for imRD. Performance was slightly lower for new diagnoses: AUC-ROC of 0.65 (95% CI 0.62-0.68) for OA ( P =0.048), 0.72 (95% CI 0.68-0.77) for FM ( P =0.157) and 0.65 (95% CI 0.60-0.69) for imRD ( P =0.034) (Figure 1 a, c, e). In the validation set, using optimized cut-offs, the models achieved high diagnostic precision with only 17% false positives for OA cases, 8% false positives for FM, and identifying 92% of imRD (Figure 1 b, d, f). Shapley additive explanations (SHAP) analysis revealed that the models identified clinically relevant linguistic patterns consistent with established diagnostic criteria for each disease type (Figure 2).

Conclusion: The analysis of free written text and the use of NLP and ML shows that non-technical descriptions of symptoms can be use in a preliminary stage to differentiate OA, FM and imRD, thereby optimising the diagnostic process and reducing waiting times. While it does not substitute for clinical assessment, this approach signifies a substantial advancement towards patient-centred methodologies. The development of precise and transparent clinical instruments will be contingent on interdisciplinary collaboration.

REFERENCES: [1] Department of Health. The musculoskeletal services framework - a joint responsibility: doing it differently. London: Department of Health; 2006. Available from: https://www.susanoliver.com/pdf/MSF_Final.pdf . Accessed on 2024 Oct 31.

[2] Picavet HSJ, Schouten JSAG. Musculoskeletal pain in the Netherlands: Prevalences, consequences and risk groups, the DMC3-study. Pain. 2003;102(1-2):167-78.

[3] Villeneuve E, Nam JL, Bell MJ, et al. A systematic literature review of strategies promoting early referral and reducing delays in the diagnosis and management of inflammatory arthritis. Ann Rheum Dis. 2013;72(1):13-22.

[4] Feuchtenberger M, Nigg AP, Kraus MR, et al. Rate of Proven Rheumatic Diseases in a Large Collective of Referrals to an Outpatient Rheumatology Clinic Under Routine Conditions. Clin Med Insights Arthritis Musculoskelet Disord. 2016;9:181-7.

[5] Wallace W, Chan C, Chidambaram S, et al. The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review. NPJ Digit Med. 2022;5(1):e667.

[6] Cooke G. A is for aphorism - is it true that “a careful history will lead to the diagnosis 80% of the time”? Aust Fam Physician. 2012;41(7):534.

Figure 1.

SVM model validation results. a,c,e) Model performance for new cases (no baseline diagnosis) vs existing cases for OA, FM, and imRD respectively. b,d,f) Swarm plots showing probability distributions for negative and positive cases. Red lines indicate high confidence thresholds; black lines show moderate confidence thresholds. For imRD (f), red line indicates direct rheumatologist referral threshold.

Figure 2.

Word stem predictive value analysis. Beeswarm plots showing positive (high values) and negative (low values) predictive influences of symptom descriptions, sorted by importance.

Acknowledgements: This SPIDeRR study was funded by a HORIZON EU Grant Agreement No 101080711.

Disclosure of Interests: None declared.

© The Authors 2025. This abstract is an open access article published in Annals of Rheumatic Diseases under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ ). Neither EULAR nor the publisher make any representation as to the accuracy of the content. The authors are solely responsible for the content in their abstract including accuracy of the facts, statements, results, conclusion, citing resources etc.