Al to Improve the Diagnosis of Rare Rheumatic Diseases (AIDRARER)

Philipps University Marburg

Status: Completed

Conditions: Rheumatic Diseases

Treatments: Other: ChatGPT

Study type: Interventional

Funder types: Other

Identifiers

NCT06748170

24-221 ANZ

Details and patient eligibility

About

This trial aims to assess the impact of providing medical students with access to ChatGPT, a state-of-the-art large language model, in comparison to conventional diagnostic decision support tools, on their diagnostic accuracy for rare rheumatic diseases.

Full description

Advanced artificial intelligence (AI) technologies, particularly large language models such as OpenAI's ChatGPT, hold significant potential for enhancing medical decision-making. While ChatGPT was not specifically designed for medical applications, it has shown utility in various healthcare scenarios, including answering patient inquiries, drafting medical documentation, and aiding consultations. Despite these advancements, its role in supporting diagnostic reasoning-especially among less experienced medical students-and for complex rare diseases remains underexplored.

Diagnostic reasoning is a multifaceted process that combines pattern recognition, knowledge synthesis, and probabilistic thinking. Tools like ChatGPT could potentially alleviate cognitive burden, enhance diagnostic accuracy, and ultimately accelerate the diagnosis for rare diseases. However, ChatGPT is not tailored for diagnostic reasoning and lacks comprehensive validation in this domain. Additionally, it is susceptible to generating misinformation or plausible-sounding but inaccurate responses, which may hinder rather than support clinical decision-making. Therefore, understanding how medical students utilize such AI tools is essential before they are integrated into educational or clinical workflows. This study will also assess a standardized prompt to facilitate ChatGPT usage and will give students direct access to enable a realistic scenario.

This study will investigate the impact of ChatGPT on the diagnostic accuracy of medical students when tackling cases of rare rheumatic diseases. Participants will be randomized into two groups: one with access to ChatGPT and one using conventional diagnostic tools. Each participant will analyze diagnostic cases by providing up to 5 differential diagnoses and and rating the diagnostic confidence. Independent reviewers, blinded to group allocation, will evaluate the accuracy and quality of their responses. This study hence aims to provide insights into the potential benefits and limitations of integrating AI tools like ChatGPT.

Enrollment: 68 patients

Sex: All

Ages: 18+ years old

Volunteers: Accepts Healthy Volunteers

Inclusion criteria

• Medical students having started with clinical subjects (Internal medicine)

Exclusion criteria

• Not being a medical student

Trial design

Primary purpose: Diagnostic

Allocation: Randomized

Interventional model: Parallel Assignment

Masking: Single Blind

---

68 participants in 2 patient groups

Intervention group

Active Comparator group

Description:

Group will be given access to ChatGPT and standardized initial prompt

Treatment:

Other: ChatGPT

Control group

No Intervention group

Description:

Group will not be given access to any LLMs including ChatGPT but will be motivated to use other resources (such as online search enginges, Pubmed)