Background: Despite patients’ critical need to know “Will this medication work for me?” clinicians lack tools to predict treatment success. While methotrexate (MTX) remains the cornerstone first-line therapy, selecting the optimal second DMARD after MTX-failure continues to be a major challenge.

Objectives: To build models that predict second-line DMARD persistence for one year using disease-, treatment-, and patient-characteristics from real-world data

Methods: We analysed data from 1,930 patients starting a second-line DMARD after failure of MTX monotherapy in Stockholm (Sweden), Leiden (the Netherlands),and Vienna (Austria), all with at least one-year follow-up. We split the data into training- (80%) and test datasets (20%). We built seven machine learning models (K-Nearest Neighbour, logistic regression, support vector classifier, random forest, gradient boosting forest, multi layer perceptron, and a stacked model of random forest and logistic regression), with hyperparameters optimised using repeated 5-fold cross-validation on the training data and applied to full data for stability. The models contained the following predictors at time of start of second-line dmard: age, sex, seropositivity status, duration of MTX monotherapy, treatment strategy (addition vs. switch of a second-line DMARD), glucocorticoid use, type of second DMARD (biological vs. conventional synthetic), inflammatory state based on low, moderate, or high crp/ESR.

Feature importance was assessed using Gini importance from the random forest model and odds ratios from logistic regression to compare predictor impact across model types on the test data. The best-performing models were combined in a stacked ensemble approach. The best-performing model was subsequently evaluated using repeated cross-validation on the full dataset, stratified by medication, with ROC curves generated for the most commonly used treatments.

Results: Of the 1,930 patients in the cohort, 1,256 remained after excluding those with missing values. Machine learning models demonstrated limited predictive performance for treatment persistence (Figure 1A), with median AUC values of 0.55–0.60 across all classifiers. Performance varied by medication (Figure 1B): etanercept (AUC=0.66) and leflunomide (AUC=0.63) showed the best predictability, while sulfasalazine performed at chance level (AUC=0.49). Feature importance differed between models: random forest prioritized joint counts (tjc28, sjc28) and the dmard type based on Gini importance (Figure 2A), whereas logistic regression (Figure 2C) identified DMARD type and age as more influential predictors. For the random forest, the removal of features was only slightly impacting the accuracy. Cross-validated odds ratios remained modest with wide confidence intervals.

Conclusions: Of the 1,930 patients in the cohort, 1,256 remained after excluding those with missing values. Machine learning models demonstrated limited predictive performance for treatment persistence (Figure 1A), with median AUC values of 0.55–0.60 across all classifiers. Performance varied by medication (Figure 1B): etanercept (AUC=0.66) and leflunomide (AUC=0.63) showed the best predictability, while sulfasalazine performed at chance level (AUC=0.49). Feature importance differed between models: random forest prioritized joint counts (tjc28, sjc28) and the dmard type based on Gini importance (Figure 2A), whereas logistic regression (Figure 2C) identified DMARD type and age as more influential predictors. For the random forest, the removal of features was only slightly impacting the accuracy. Cross-validated odds ratios remained modest with wide confidence intervals.

REFERENCES: NIL.

Acknowledgments: NIL.

Disclosure of Interests: Nils Steinz: None declared, Daniela Di Giuseppe: None declared, Claudia Anna Hana: None declared, Helga Lechner-Radner Abbvie, Amgen, J&J, Sanofi, Eli Lilly, Merck Sharp & Dohme, UCB, Pfizer, Fresenius-Kabi, Johan Askling Abbvie, BMS, Galapagos, MSD, Pfizer, Roche, Samsung Bioepis, Sanofi, Rachel Knevel: None declared.