fetching data ... 
Background: Systematic literature reviews (SLRs) are a core methodological component of many EULAR activities and provide the foundation for rigorous and transparent evidence synthesis. However, SLRs have become increasingly time- and labor-intensive in the ever-growing research landscape of modern rheumatology, and the comprehensive synthesis of large data volumes has become a major challenge for researchers. Therefore, we explored using artificial intelligence (AI) and machine learning (ML) tools to improve efficiency and reduce the resources needed to perform SLRs, in this case specifically focusing on title and abstract screening.
Objectives: To assess the performance of different ML algorithms for title and abstract screening in EULAR SLRs on clinical trials and observational studies, in terms of accuracy compared to conventional manual screening, as well as usability, transparency and data privacy.
Methods: Four SLRs were conducted to inform the 2025 update of the EULAR management recommendations for rheumatoid arthritis (RA) with disease-modifying antirheumatic drugs (DMARDs), all following EULAR standardized operating procedures. Title and abstract screening for these SLRs was subsequently replicated with different ML tools; with their performance assessed against the previously conducted manual screening, which served as the gold standard. Eligible algorithms were identified using the Systematic Review Toolbox, a comprehensive overview of software related to SLR conduct [1], complemented by online searches and reference checking. Actively maintained (in the past 5 years) and readily accessible ML tools that offered title and abstract screening capabilities beyond those of commonly used reference managers were included.
Results: Nine ML models that implement active learning via record prioritization and relevance prediction in response to user feedback were identified, of which three were evaluated in detail, as they fulfilled all inclusion criteria. A total of 12,762 records were manually screened by two researchers (VK, FL) for four SLRs on different topics: i) efficacy of DMARDs, ii) safety of DMARDs, iii) DMARD treatment of RA-associated interstitial lung disease, iv) DMARD treatment in preclinical RA. A total of 132 articles were finally included. When replicating the screening with different ML models, a mean overall reduction of workload of 77.8% ± 12.8% could be achieved; 2071 of 12,762 records (16.2%) had to be screened with the ML model “ASReview”, 4705 (36.9%) with “SWIFT Active Screener”, and 1705 (13,4%) with “Research screener”. When using the manual search as a gold standard, ASReview missed 3 (2.3%), achieving a recall of 97.7%, SWIFT Active Screener also missed 3 (2.3%), and Research Screener missed 2 (1.5%) articles, achieving a recall of 98.5%; with higher accuracy observed for SLRs that considered randomized-controlled trials (RCTs) only, compared with those that also included observational studies. Different articles were missed by each tool, and no specific pattern was detectable. ML was further used for classification and filtering of RCTs, meaning that non-RCT studies can be filtered out before starting the manual screening process. The Cochrane RCT filter, an RCT classifier developed and validated in large Cochrane and Clinical Hedges datasets, enabled a screening workload reduction of 57.3% ± 7.4% (mean number of abstracts that needed to be screened after applying the filter, across all SLRs that consider only RCTs). Table 1 summarizes the ML models that were evaluated in detail for the current project. An overview of workload savings as well as model performance is provided in Figure 1 , which shows the number of records that were screened manually compared with the three ML models, as well as the number of correctly identified and missed articles for each SLR. Recall curves further depict screening efficiency, measured by the proportion of records identified with the ML models compared with the manual screening. Apart from accuracy and efficiency, the tools differed regarding their usability, transparency and aspects related to data protection: AS Review constitutes an open-source and locally deployable software that required basic technical setup, whereas SWIFT Active Screener and Research Screener offer easily implementable cloud-based (online) solutions, with corresponding trade-offs in terms of transparency, data privacy and data control.
Conclusions: ML models have high potential to support the conventional manual title and abstract screening process performed by human researchers. However, this project also highlighted important limitations and pitfalls associated with these models, such as ML generated errors, the amplification of human error in the active learning processes, the potential loss of human experience in SLR conduct and methodology, as well as a possible inflation of low-quality or biased research when ML is applied poorly. Prospective studies are needed to delineate effective and accurate strategies for implementing ML models in future EULAR task force projects.
Table 1. Machine learning models explored in the current project
Screening results (manually conducted search vs machine learning-supported searches )
REFERENCES: [1]
Acknowledgments: NIL.
Disclosure of Interests: Victoria Konzett received lecture fees from Eli Lilly, AbbVie and AstraZeneca, received consulting fees from Eli Lilly, AbbVie, Alfasigma and J&J, Faidra Laskou received lecture fees from Alfasigma, Josef S. Smolen received lecture or consulting fees from Abbott, Abbvie, Anaptys, Ananda, Astra-Zeneca, Celltrion, Chugai, Immunovant, Lilly, R-Pharma, Samsung, Sandoz, Shattuck labs and UCB, Christopher John Edwards received honoraria, advisory boards, speaker’s bureau, research support from Abbvie, Alphasigma, Astra Zeneca, BMS, Biogen, Boehringer Ingelheim, Celgene, Celltrion, Fresenius, Gilead, GSK, Janssen, Lilly, Pfizer, MSD, Novartis, Roche, Samsung, Sanofi and UCB, Daniel Aletaha received honoraria from Advanz, Janssen/Johnsen & Johnsen, Sanofi, Mitsubishi Tanabe, UCB and AstraZeneca, received grants to his institution from Lilly and Johnson & Johnson, Désirée van der Heijde received consulting fees from AbbVie, Alfasigma, ArgenX, BMS, Elly-Lilly, Grey-Wolf Therapeutics, Janssen, Novartis, Pfizer, Takeda and UCB Pharma, Kevin L. Winthrop received consulting fees from AbbVie, AstraZeneca, BMS, Eli Lilly, Galapagos, Gilead, GlaxoSmithKline (GSK), Moderna, Novartis, Roche, Sanofi, Union Chimique Belge (UCB), Zurabio, Takeda, Ouros and Otsuka, Tsutomu Takeuchi received speaker’s or consultancy fees from AbbVie GK, Chugai, Eli Lilly Japan, Eisai, Gilead Sciences Inc, Pfizer Japan Inc, Mitsubishi-Tanabe and Taisho Pharma, Roberto F. Caporali received speaker’s or consulting fees from AbbVie, Alfasigma, Astra-Zeneca, GSK, Eli Lilly, Galapagos, Janssen, Novartis, Pfizer and UCB, Patrick Verschueren received speaker’s or consultancy fees from AlfaSigma, Abbvie, Galapagos, Eli Lilly, Cytrill and Boehringer Ingelheim, PV received research grants from AlfaSigma and Pfizer, Janet Pope: None declared, Kimme Hyrich received speaker’s fees from Abbvie, received grants from Pfizer and BMS, Savia de Souza: None declared, Tanja Stamm: None declared, Jette Primdahl received speaker’s fees from Pfizer and Eli Lilly, Jan Schoones: None declared, Robert Landewé received lecture or consulting fees from AbbVie, Jansen Pharma, Galapagos, Novartis and UCB, Andreas Kerschbaumer received lecture fees from Lilly, Galapagos, Janssen, MSD, Novartis and Pfizer, received consulting fees from Abbvie, Lilly, Gilead, Janssen and UCB.