AI-Orchestrated Workflow Versus Consultant Ophthalmologist for Refractive Surgery and Keratoconus Diagnosis (AEYE Trial)

Hazem Yassin Clinics

Status: Invitation-only

Conditions: Diagnostic Accuracy; Keratoconus; Ophthalmology; Machine Learning; Artifical Intelligence; Clinical Decision Support; Refractive Surgery

Treatments: Diagnostic Test: Multi-Agent AI Diagnostic Workflow (AEYE)

Study type: Observational

Funder types: Other

Identifiers

NCT07096232

HYC-AI-RS-2025-01

Details and patient eligibility

About

Background and Rationale:

Laser vision correction procedures, such as LASIK (Laser-Assisted In Situ Keratomileusis), PRK (Photorefractive Keratectomy), and SMILE (Small Incision Lenticule Extraction), are highly effective but require careful preoperative screening to ensure safety. One of the most critical aspects of screening is identifying keratoconus and other corneal ectatic disorders-conditions that cause progressive thinning and bulging of the cornea, often contraindicating surgery. Early detection is essential to avoid vision-threatening complications.

Despite advanced corneal imaging tools such as Scheimpflug tomography and anterior segment optical coherence tomography (AS-OCT), accurate diagnosis-particularly in borderline or early-stage cases-remains challenging and subject to variability in human interpretation. Artificial intelligence (AI) offers the potential to improve diagnostic precision, reduce oversight, and standardize surgical planning.

Purpose of the Study:

This study evaluates the performance of AEYE (Automated Evaluation for Your Eye), a multi-agent AI system designed to support ophthalmologists in diagnosing keratoconus and determining refractive surgery eligibility. AEYE simulates the clinical workflow of an anterior segment specialist by orchestrating three specialized agents:

History & Risk Agent: Reviews patient history and extracts risk factors.

Imaging Agent: Analyzes corneal tomography, AS-OCT, and epithelial mapping scans.

Surgical Decision Agent: Integrates all findings, assigns a diagnosis, and recommends appropriate treatment options, including surgical eligibility or corneal cross-linking (CXL).

Study Design:

The study includes 50 real-world patient cases, both retrospective (from 2020 onward) and prospective, who were evaluated for refractive surgery or keratoconus. Each case is analyzed independently by AEYE and a consultant ophthalmologist (blinded to AI output), using the same multimodal clinical and imaging data. Diagnostic accuracy, agreement in surgical recommendations, and workflow efficiency are assessed.

Anticipated Impact:

By comparing AI-derived decisions with expert clinical judgment, this study aims to validate whether structured AI workflows like AEYE can serve as reliable, safe, and explainable decision support tools. If successful, AEYE may offer a scalable solution to reduce diagnostic variability and enhance the safety and consistency of refractive surgery screening.

Full description

Technical Protocol Summary

This is a diagnostic performance study evaluating AEYE (Automated Evaluation for Your Eye), an orchestrated multi-agent artificial intelligence (AI) system designed to assist ophthalmologists in the screening and management of keratoconus and refractive surgery planning. AEYE combines large language models (LLMs) with deterministic code logic to replicate and support the clinical decision-making process typically performed by anterior segment specialists.

System Architecture and Workflow

AEYE is structured as a modular pipeline of three specialized agents, each focused on a distinct diagnostic task:

History and Risk Agent: Extracts structured data from unstructured clinical records, including demographics, ocular/systemic history, medication use, and risk factors relevant to keratoconus or refractive surgery eligibility.

Imaging Analysis Agent: Processes multimodal anterior segment imaging such as Scheimpflug-based tomography (e.g., Pentacam), anterior segment optical coherence tomography (AS-OCT), and epithelial thickness mapping. It standardizes key metrics like maximum keratometry (Kmax), thinnest corneal pachymetry, anterior/posterior elevation, Belin-Ambrósio Deviation Index (BAD-D), and Pachymetric Progression Index (PPI). Each eye and each imaging file is processed independently to avoid misattribution.

Surgical Decision Agent: Integrates the outputs of the previous agents to generate a final diagnosis, assign keratoconus staging (e.g., ABCD, Amsler-Krumeich classification), and recommend next steps, such as LASIK (Laser-Assisted In Situ Keratomileusis), PRK (Photorefractive Keratectomy), SMILE (Small Incision Lenticule Extraction), phakic intraocular lenses (ICL), or corneal collagen cross-linking (CXL). The output is a structured, auditable report.

All agents are controlled by a deterministic workflow using Python scripts for data merging, schema validation, and output formatting. Structured memory is maintained using JSON objects that store the full diagnostic context per patient. The system is designed to ensure reproducibility, reduce human variability, and support explainable clinical decision-making.

Study Workflow and Scope

Fifty real-world patient cases are included, comprising both retrospective records (from January 2020 onward) and newly enrolled prospective cases. Each case includes comprehensive clinical data and anterior segment imaging. AEYE analyzes the case and generates a structured report. Separately, an experienced ophthalmologist (blinded to the AI output) reviews the same data and records their clinical decisions.

Key metrics include:

Diagnostic accuracy of keratoconus detection. Agreement in surgical eligibility assessments. Efficiency of workflow execution. Variability in results across different LLMs used in agent roles. Cases with discordant results may undergo adjudication to establish a reference standard.

Innovation and Clinical Relevance.

AEYE represents a novel application of explainable AI in ophthalmology. Its multi-agent design reflects a divide-and-conquer strategy, reducing cognitive load on any single model while enforcing clinical safety through deterministic logic. The system supports scalability, modularity, and integration into electronic health record (EHR) systems. The study will help determine whether AEYE can function as a safe, consistent, and effective assistant in complex diagnostic pathways for corneal ectatic disease.

Enrollment: 50 estimated patients

Sex: All

Volunteers: No Healthy Volunteers

Inclusion criteria

1. Diagnosis of Refractive Error or Keratoconus:

   • Patients with a confirmed clinical diagnosis of refractive errors (myopia, hyperopia, or astigmatism) or keratoconus, as determined by a qualified ophthalmologist based on current diagnostic standards.

2. Availability of Complete Data:

   • Clinical records must include comprehensive demographic data, medical and ophthalmic history, refraction measurements, and high-quality corneal imaging (e.g., corneal topography, tomography, or OCT).

3. Eligible for Both AI and Consultant Review:

   • Patients must have data sets that allow both the multi-agent AI workflow (AEYE) and an independent consultant ophthalmologist to conduct a complete diagnostic assessment.

4. Consent:

   • For prospectively enrolled patients, written informed consent must be obtained prior to participation. For retrospective cases, waiver of consent may be granted according to local IRB/ethics committee policies.

5. No Restriction on Age or Sex:

   • There are no specific age or sex limitations for inclusion in this study. Pediatric and adult cases may be included if data are available and consent is appropriately obtained.

Clinical Documentation Requirements:

• Availability of complete ophthalmic records including:
• Uncorrected and best corrected visual acuity (UCVA, BCVA)
• Manifest refraction (sphere, cylinder, axis)
• History of contact lens use and duration
• Prior ocular surgery or trauma, if applicable
• Family history of keratoconus or corneal ectasia
• Use of medications that may influence tear film stability or corneal biomechanics
• Ocular surface disease documentation if present (e.g., dry eye, blepharitis)

Imaging and Diagnostic Data Requirements:

• Availability of at least one valid and complete set of:
• Scheimpflug-based corneal tomography (e.g., Pentacam or equivalent)
• Anterior segment OCT (AS-OCT), if performed
• Epithelial thickness mapping (optional but desirable)
• Corneal biomechanical data (optional)
• Imaging files must be:
• Of adequate quality as judged by the original technician and reviewer
• Free from significant artifacts or motion blur
• Properly labeled by eye (OD/OS) and imaging modality
• Acquired with standard scan protocols

Diagnostic Spectrum Requirements:

• Patients may fall into any of the following diagnostic categories:
• Normal cornea eligible for laser refractive surgery
• Suspect keratoconus or forme fruste keratoconus
• Established keratoconus of any stage (I-IV or ABCD)
• Post-CXL (cross-linking) cornea undergoing follow-up evaluation
• Post-keratoplasty cornea (e.g., DALK, PKP) undergoing diagnostic review
• Patients disqualified from surgery due to abnormal tomography or other contraindications.

Exclusion criteria

1. Incomplete or Poor Quality Data:

   • Patients with missing, incomplete, or poor-quality clinical data or corneal imaging that precludes reliable diagnosis by either AI or consultant review.

2. Ocular Comorbidities:

   • Presence of ocular diseases that could confound the diagnosis or interpretation of refractive error or keratoconus, such as advanced glaucoma, active uveitis, significant retinal pathology, or previous corneal transplantation.

3. Severe Systemic Disease Affecting the Eye:

   • Patients with systemic diseases known to affect the cornea or refraction (e.g., connective tissue disorders with corneal involvement) will be excluded to avoid confounding effects.

4. Inability or Refusal to Consent:

   • For prospectively enrolled cases, patients (or guardians, in the case of minors) who are unwilling or unable to provide informed consent will be excluded.

5. Participation in Conflicting Studies:

   • Patients currently enrolled in other interventional studies that could interfere with the diagnostic process or outcomes measured in this study.

Clinical Documentation Exclusions:

• Missing or incomplete visual acuity data.
• Absence of manifest refraction or subjective refraction data.
• Incomplete or conflicting patient history with unresolvable discrepancies.
• Absence of necessary demographic identifiers (e.g., gender, age, laterality of findings).

Imaging and Data Quality Exclusions:

• Imaging datasets with significant artifacts that preclude AI analysis.
• Files missing key parameters (e.g., Belin/Ambrósio D index, Kmax, pachymetry) required for keratoconus classification.
• Inability to determine laterality (OD/OS) for specific scans.
• Scans acquired with outdated or non-standard imaging protocols.
• Mislabeling or duplication of imaging sets across different patients.

Case-Type Exclusions:

• Cases involving active ocular infection or inflammation at the time of evaluation.
• Cases with active herpetic eye disease, corneal ulcer, or epithelial defects interfering with imaging interpretation.
• Prior laser refractive surgery unless explicitly classified and used as part of post-refractive ectasia evaluation.
• Eyes with concurrent advanced retinal pathology that may confound diagnostic interpretation (e.g., macular edema, retinal dystrophies).

Technology-Specific Exclusions:

• Imaging performed using devices not compatible with the AEYE data ingestion framework (e.g., certain older corneal topographers).
• Non-English documentation incompatible with NLP parsing in the General Ophthalmology Agent.

Other Considerations:

• AEYE must be able to process the case within its defined schema. If key data elements are absent or the case causes schema failure, it will be excluded from quantitative analysis but may be reported qualitatively as part of system performance limitations.
• If multiple attempts at AI processing result in non-deterministic or erroneous outputs, such cases will be logged but excluded from primary outcome evaluation.
• Human reviewers (consultant ophthalmologists) must have completed their diagnostic interpretation independently prior to AEYE input; any cases where human opinion was influenced by AI output will be excluded to preserve blinding.

Trial design

50 participants in 1 patient group

Patients with refractive errors or keratoconus assessed by agentic AI workflow and consultant review

Description:

This group includes all patients with refractive errors or keratoconus evaluated at our center during the study period. Each patient case undergoes comprehensive diagnostic assessment using both an orchestrated multi-agent artificial intelligence (AI) workflow (AEYE) and independent review by a consultant ophthalmologist. The AI workflow integrates multiple specialized AI agents to analyze clinical data, imaging studies, and relevant diagnostic metrics. Results from the AI system are compared to consultant assessments to evaluate concordance, diagnostic accuracy, and workflow efficiency. Both retrospective patient records and prospectively enrolled cases are included in this group.

Treatment:

Diagnostic Test: Multi-Agent AI Diagnostic Workflow (AEYE)