Generative AI Radiologist's Workstation (GARW)

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department

Status: Not yet enrolling

Conditions: Study Aims to Develop and Validate a Generative AI-assistant Designed to Optimize Radiologists' Workflows and to Evaluate Its Performance

Study type: Observational

Funder types: Other

Identifiers

NCT07057830

2025-2

Details and patient eligibility

About

This study aims to develop a generative AI assistant for radiologists to automate the processing of electronic medical records (EMRs) and provide relevant clinical information, optimizing diagnostic interpretation workflows.

Full description

This study aims to develop and validate a generative AI-assistant designed to optimize radiologists' workflow by automatically processing electronic medical records (EMRs) and generating structured clinical summaries. The AI tool will extract and prioritize relevant patient data to support accurate and efficient interpretation of diagnostic imaging studies.

The study rationale originates from increasing radiology workloads and the need to reduce time spent reviewing EMRs while maintaining diagnostic accuracy. The proposed AI solution specifically targets these issues through advanced natural language processing capabilities, with particular attention to optimizing time efficiency while maintaining or improving diagnostic accuracy.

The study consists of 9 Stages:

Stage 1: Theoretical Foundation.

1.1 Systematic review: comprehensive analysis of existing LLM applications in radiology.

1.2 Healthcare system analysis: evaluation of LLM implementations in clinical settings.

1.3 Expert consensus: semi-structured interviews with 30 practicing radiologists (stratified by experience: junior [<3 years], mid-career [3-10 years], senior [>10 years]) to establish:

• Minimum required clinical data elements;
• Optimal summary format (structured vs. narrative);
• Critical alert thresholds.

Stage 2: Technical Development.

2.1 Medical text processing: formalization of methods for extraction, standardization, and annotation.

2.2 Dataset Curation: methodology for creating representative training datasets from UMIAS (Unified Medical Information and Analytical System).

2.3 Validation Framework: creation of validation methodology for the generative AI-based assistant.

Development and validation of a questionnaire assessing:

• Relevance;
• Completeness (missing critical data);
• Hallucination frequency;
• Terminology/grammar;
• Radiologist satisfaction.

Stage 3: Dataset Development.

Data Source: Retrospective extraction of anonymized EMRs from UMIAS.

Inclusion Criteria:

• Age of the patient at the moment of medical image acquisition >18 years;
• Pathology types: pleuritis, ascites, unspecified masses/lesions, neuropathy;
• Imaging study modalities (performed between January 1, 2020, and May 31, 2025): CT chest (pleural/pulmonary pathologies); CT abdomen/pelvis (ascites/abdominal masses); MRI brain (neuropathy/neurological conditions);
• Complete EMR data (clinical notes, prior imaging reports, lab results, discharge summaries).

Exclusion Criteria:

Cases with technical artifacts on medical images compromising diagnostic quality.

Per-case data collected: physical examination results; two prior imaging reports (same modality) for progression assessment; three laboratory test results; consultation notes from three clinical specialists; discharge summaries; AI-Generated summaries (three summaries of different quality), including:

• High quality summary: complete, well-structured, clinically relevant;
• Medium quality summary: partial omissions, acceptable structure;
• Low quality summary: significant omissions/poor structure.

Stage 4: Comparative analysis of open-license generative AI architectures.

Stage 5: Model selection according to pre-defined selection criteria.

Stage 6: Model adaptation (fine-tuning and prompt optimization).

Stage 7: Development and UMIAS integration of a minimum viable product (MVP).

Stage 8: Pilot Testing.

Participants: 27 radiologists divided into three groups (A, B and C; n=9 each). Detailed description of each group is in section 'Groups and Interventions'. The group B will evaluate AI-summary quality via specially developed and validated questionnaire (scores: ≤8=low, 9-15=medium, >15=high).

In the end of pilot testing primary and secondary outcomes will be assessed.

Stage 9: Comparative analysis across all groups. Formulation of conclusions and assessment of the AI-assistant's applicability.

Enrollment: 27 estimated patients

Sex: All

Ages: 18+ years old

Volunteers: No Healthy Volunteers

Inclusion criteria

• Board-certified practicing radiologist;
• ≥3 years of experience in diagnostic imaging;
• Proficiency in using UMIAS systems;
• Signed informed consent form.

Exclusion criteria

• Participation in other studies;
• Unwillingness to adopt new technologies in daily practice;
• Conflict of interest.

Trial design

27 participants in 3 patient groups

Group A: Reviews full electronic medical record without AI summaries

Description:

Group A will review full electronic medical records without AI-generated summaries. Participants will be required to determine the purpose of the radiological examination and prepare a full radiology report (including protocol and conclusion).

Group B: Evaluates AI summaries via validated questionnaire

Description:

Group B will receive access to full electronic medical records and to AI-generated summaries, which they will have to evaluate via specially developed and validated questionnaire. Participants will have to determine the purpose of the radiological examination, generate a complete radiology report (protocol + conclusion) and evaluate the AI summaries using a validated questionnaire.

Group C: Receives AI summaries only

Description:

Group C participants will receive only AI-generated clinical summaries without access to full electronic medical records. Each radiologist in this group will be required to determine the purpose of the radiological examination and generate a complete radiology report consisting of both protocol documentation and diagnostic conclusion. Comparative analysis will be performed against Groups A and B for all measured outcome parameters.