Physician Diagnosis of Neurologic Cases With Large Language Models

Capital Medical University

Status

Completed

Conditions

Neurology

Treatments

Other: LLM assistance

Study type

Interventional

Funder types

Other

Identifiers

NCT07344792

LLM-001

Details and patient eligibility

About

This study is a prospective randomized controlled trial that aims to evaluate whether large language model (LLM) assistance improves physicians' diagnostic performance compared with conventional assistance.

Full description

The study population consists of licensed physicians who have completed at least two years of neurology training. Participants are randomly assigned in a 1:1 ratio to the LLM assistance group or the conventional assistance group. In the LLM assistance group, physicians are assisted by an in-house LLM and are allowed to use conventional resources (e.g., search engines, UpToDate, or clinical guidelines) before submitting a final diagnosis. Access to any other LLMs is not permitted. In the conventional assistance group, physicians use only conventional resources and do not have access to the in-house LLM or any other LLMs at any point during the study. The primary outcome is top-1 diagnostic accuracy. Secondary outcomes include top-3 diagnostic accuracy, the time required to complete each case, and physicians' diagnostic confidence.

Enrollment

40 patients

Sex

All

Volunteers

Accepts Healthy Volunteers

Inclusion criteria

Licensed physicians who have completed at least two years of neurology training.
Willing to participate in the study and able to provide written informed consent.

Exclusion criteria

Physicians who have previously participated in the development of the in-house LLM or the construction of datasets used in this study.
Physicians who were involved in the clinical care of patients whose cases were used to construct the case dataset for this study.
Physicians who are known to have previously reviewed or been exposed to the specific cases used in this study.