Evaluation of a Retrieval Augmented Large Language Model as a Diagnostic Copilot in Rheumatology (ALLIANCE)

Philipps University Marburg

Status: Not yet enrolling

Conditions: Diagnosis

Treatments: Other: Prof. Valmed

Study type: Interventional

Funder types: Other

Identifiers

NCT07166692

24-221-1

Details and patient eligibility

About

This trial evaluates whether providing physicians with access to Prof. Valmed, a clinical decision support medical product, improves identification of rheumatic diseases and formulation of differential diagnoses compared with conventional decision support.

Full description

Advanced AI, particularly large language models, shows promise for enhancing clinical reasoning, yet most systems such as ChatGPT are not certified as medical products. Prof. Valmed is a clinical decision support medical product designed to assist physicians in diagnostic decision making. Given frequent referral problems and diagnostic delays in rheumatology, evaluating such support is highly relevant for clinical workflows.

This randomized controlled trial will test whether access to Prof. Valmed improves physicians' diagnostic performance in cases of suspected rheumatic disease compared with conventional decision support. Participants will be randomized to either use Prof. Valmed or rely on conventional tools while working through standardized clinical cases. For each case, participants will submit up to three differential diagnoses and a confidence rating. Independent reviewers, blinded to group allocation, will adjudicate accuracy. Findings will clarify the benefits and limitations of integrating Prof. Valmed into routine practice.

Enrollment: 82 estimated patients

Sex: All

Volunteers: Accepts Healthy Volunteers

Inclusion criteria

• Participants must be licensed physicians.
• Training in rheumatology, internal medicine, emergency medicine, family medicine, dermatology or orthopedics.

Exclusion criteria

• Not currently practicing clinically.

Trial design

Primary purpose: Diagnostic

Allocation: Randomized

Interventional model: Parallel Assignment

Masking: Single Blind

---

82 participants in 2 patient groups

Prof. Valmed

Experimental group

Description:

Group will be given access to Prof. Valmed.

Treatment:

Other: Prof. Valmed

Control group

No Intervention group

Description:

Group will not be given access to Prof. Valmed but will be encouraged to use any resources they wish besides large language models (UpToDate, etc).