Background: Patients with rheumatic diseases have complex, lifelong information needs, yet access to reliable, guideline-concordant answers in routine care remains limited. Consequently, patients are often left to navigate lengthy, difficult-to-understand documents and to judge the quality of health information on their own. Artificial intelligence–based chatbots represent a promising, scalable approach to patient education; however, evidence on their real-world use and patient experience is still scarce.

Objectives: To develop disease-specific, guideline-based chatbots for patients with rheumatic diseases and to evaluate their real-world use and patient-reported experience.

Methods: In collaboration with two patient research partners from the German League Against Rheumatism, ten disease-specific chatbots covering distinct rheumatic diseases were developed (Figure 1A). Each chatbot was grounded in the respective current German clinical guideline using a retrieval-augmented generation (RAG) approach to minimize hallucinations. Guideline content was hierarchically segmented and embedded into a vector database using a fine-tuned, domain-specific large-context embedding model, enabling semantic retrieval of the most relevant passages for a given user query. Top-matching excerpts were injected as contextual grounding into the model prompt, ensuring alignment with guideline content while improving robustness to paraphrased questions. ChatGPT-4o served as the base model, with predefined system prompts enforcing patient-appropriate language. The chatbots were promoted via patient organizations and rheumatologists, including QR codes on informational leaflets. Users could ask disease-related questions and provide immediate feedback on individual chatbot responses. After completing a session, participants were invited to complete an evaluation questionnaire assessing usability, usefulness, trust, and perceived response quality. The Ethics Committee of Philipps University Marburg confirmed that formal ethical approval was not required for this anonymous survey study (reference: 25-237-Anz).

Results: Between 2 September 2025 and 3 January 2026, a total of 5131 chatbot interactions (question–answer pairs) were recorded across 1312 individual chatbot sessions. Direct feedback was provided for 2165 chatbot answers, of which 2012 (92.9%) received a “like” and 153 (7.1%) a “dislike” (Figure 1A). A total of 520 patients completed the evaluation questionnaire; 489 (94.0%) reported a diagnosed rheumatic disease, most commonly rheumatoid arthritis (RA, n=213), axial spondyloarthritis (axSpA, n=106), and systemic lupus erythematosus (SLE, n=49). The mean (SD) age was 49.2 (13.5) years. Prior use of AI-based tools for health-related questions was reported by 41% of participants, and 63% perceived AI in healthcare as positive or very positive. Overall, 86% (strongly) agreed that the chatbot was easy to use and that its answers were easy to understand, while 82% considered it a useful addition to existing patient education materials. Furthermore, 79% indicated that the chatbot would save time when searching for answers, 75% rated the answer quality as at least acceptable, 69% found the answers trustworthy, and 58% preferred using the chatbot over general internet searches (e.g. Google), see Figure 1B.

Conclusions: Disease-specific, guideline-based chatbots were well received by patients with rheumatic diseases and showed high levels of usability, perceived usefulness, and trust. Future work will focus on systematic scientific validation of chatbot responses, scalable maintenance and error correction strategies, and evaluation of pathways for routine use in patient education and clinical care.

Figure 1.

Distribution of patient ratings of chatbot answers expressed as percentages and like:dislike ratios (A), and responses to the chatbot evaluation questionnaire (B).

REFERENCES: NIL.

Acknowledgments: NIL.

Disclosure of Interests: Tim Wilhelmi: None declared, Vanessa Bartsch: None declared, Marian Weber Zentrum für Telemedizin Bad Kissingen, Robert Orzanna Zentrum für Telemedizin Bad Kissingen, Asarnusch Rashid Zentrum für Telemedizin Bad Kissingen, Johannes Hornig: None declared, Martin Krusche: None declared, Axel Hueber: None declared, Daniel Fink: None declared, Alexander Pfeil: None declared, Gabriel Dischereit: None declared, Marie-Therese Holzer: None declared, Ulrich Drott: None declared, Peer Malte Aries: None declared, Max Müller: None declared, Peter Böhm: None declared, Sebastian Kuhn: None declared, Philipp Klemm: None declared, Johannes Knitza Abbvie, AstraZeneca, BMS, Boehringer Ingelheim, Chugai, Fraunhofer, Fachverband Rheumatologische Fachassistenz, GAIA, Galapagos, GSK, Janssen, Lilly, Medac, Novartis, Pfizer, Rheumaakademie, Sanofi, Sobi, UCB, Vila Health, Abbvie, Chugai, GAIA, GSK, Abbvie, AlfaSigma, Deutsche Rheumastiftung, GSK, Vila Health.