Can Feedback From a Large Language Model Improve Health Care Quality?

Yale University

Status: Completed

Conditions: All Conditions

Treatments: Other: Large Language Model Clinical Decision Support

Study type: Interventional

Funder types: Other

Identifiers

NCT06823765

2000035990

Details and patient eligibility

About

The goal of this study is to learn if computer-assisted advice can help improve patient care in Nigerian health clinics. The main question it aims to answer is: does giving healthcare workers instant computer feedback help them make better decisions about patient care?

Researchers will compare patient care notes written by healthcare workers before and after they receive computer feedback to see if the feedback improves care quality. A doctor who doesn't know if feedback was given will review these notes.

Participants will:

• Be seen by a community healthcare worker who uses the computer feedback system
• Be treated by a fully trained medical doctor
• Get tested for malaria, anemia, or urinary tract infections if they have certain symptoms

Full description

This project tests whether Large Language Models (LLMs) can improve patient care in Nigerian primary care clinics by giving customized and instant feedback to the provider in natural language. An LLM-based tool integrated into an electronic patient record management system provides "second opinions" to community health extension workers (CHEWs) at two clinics in Nigeria. These second opinions are intended to mirror what a reviewing physician might advise the CHEWs after seeing or hearing their initial report on a patient.

For the main analysis, this study employs a within-patient comparison of two patient notes created by the CHEW; one during the initial patient consultation, and one after the LLM feedback was received. The patient is also seen by a fully trained medical officer who is in charge of patient care. The MO conducts a blinded review of the CHEW's patient notes to measures changes in the CHEW's care as a result of the LLM feedback. The data comes from the information captured in the electronic medical record (EMR) of the patient and from survey data collected from CHEWs, reviewing MOs, and a panel of reviewing Medical Doctors.

Enrollment: 491 patients

Sex: All

Volunteers: No Healthy Volunteers

Inclusion criteria

• Patient is at the clinic for outpatient consultation
• Parent/guardian consent is required for individuals under 18

Exclusion criteria

• Patient does not require emergency care
• Patient is not at the clinic for a checkup (e.g. weight, blood pressure, follow up after recovery)
• Patient is not a trauma patient (visit is not for an accident, wound or injury)
• Patient is not at the clinic for a scheduled procedure or a birth

Trial design

Primary purpose: Health Services Research

Allocation: N/A

Interventional model: Single Group Assignment

Masking: None (Open label)

---

491 participants in 1 patient group

Clinical Assessment with and without LLMs

Experimental group

Description:

The investigators employ a within-patient design. Patients receive two sequential assessments from a Community Health Extension Worker: first without and then with Large Language Model assistance.

Treatment:

Other: Large Language Model Clinical Decision Support