Effectiveness of Original vs. AI-Generated Plain Language Summaries of Systematic Reviews

University of Split

Status: Invitation-only

Conditions: Non-Metastatic Breast Cancer

Treatments: Behavioral: Original PLS; Behavioral: AI-generated PLS

Study type: Interventional

Funder types: Other

Identifiers

NCT07140107

2181-198-03-04-25-0071

2181-147/01-06/LJ.Z.- 25-02 (Other Identifier)

Details and patient eligibility

About

The study explores how patients with non-metastatic breast cancer perceive and comprehend original Plain Language Summary and AI-generated Plain Language Summary (PLS) of two systematic reviews from the Cochrane Library. The systematic reviews address exercise during adjuvant therapy and psychological interventions for non-metastatic breast cancer. The study, which is randomized and double-blind, aims to assess whether AI-generated PLS and original PLS differ in their influence on patients' intention to undertake and apply the recommendation presented in the summary, as well as their overall understanding of the study results.

Full description

Studies show that Plain Language Summaries (PLS), which are intended to improve accessibility to health information for laypeople, are lacking in terms of comprehension, readability, and user engagement. Further empirical research is needed to evaluate the comparative effectiveness of AI-generated summaries versus original Plain Language Summaries. This study, conducted at the Department of Oncology and Radiotherapy at the University Hospital of Split aims to explore whether large language models (LLMs), such as ChatGPT, can be leveraged to provide summaries of complex medical content, as those found in systematic reviews, to be more streamlined and adapted for laypeople. The prospective, randomized, single-center, parallel group, double-blind, controlled trial focuses specifically on patients diagnosed with non-metastatic breast cancer, histologically confirmed, and aims to help them make more evidence-based medical decisions regarding their health and recovery. The study's primary objective is to analyze patients' level of intention for changing their behavior and start applying the interventions described in the summary. The level of intention to change behavior will be measured using a visual analog scale ranging from 0 to 100 (0 - do not agree at all and 100 - fully agree). The secondary outcomes are patients' level of understanding calculated by the score received on a five-question knowledge survey (5 as a maximum score), the level of trust in (range 0-100) and clarity of (range 0-100) the information presented in the PLS, and their confidence in their decision to change behavior (range 0-100). The survey is based on the information presented in the PLS and the questions focus on patients' understanding of the benefits and risks of the intervention as well as the quality of evidence described in the systematic reviews. Recruitment for this study will take place over 6 months. Patients will be invited to participate in the study before or after their routine oncology appointment. The participants will read two summaries from two respective Cochrane reviews: exercise during adjuvant therapy and psychological interventions for non-metastatic breast cancer. They will then take the brief survey that tests the impact and likelihood of applying the recommendations described in the summary and their overall understanding of the medical information discussed in the reviews. The study includes female adult oncology patients diagnosed with non-metastatic breast cancer, histologically confirmed, following a modality of the treatment, which includes one or more of the following: neoadjuvant chemotherapy (with or without immunotherapy or antiHER2 therapy); adjuvant chemotherapy (with or without immunotherapy or antiHER2 therapy); adjuvant radiotherapy (including local and locoregional radiation treatment, with possible radiotherapy boost where indicated). Since participants in this study are limited to reading material and completing a survey, the study is non-invasive in nature and poses low-risk. Data will be analyzed using descriptive statistics to summarize participant characteristics and assess baseline comparability between the two study arms.

Enrollment: 130 estimated patients

Sex: Female

Ages: 18+ years old

Volunteers: No Healthy Volunteers

Inclusion criteria

• female patients with histologically confirmed non-metastatic breast cancer currently undergoing chemotherapy or radiotherapy:

  1. neoadjuvant chemotherapy (with or without immunotherapy or antiHER2 therapy),
  2. adjuvant chemotherapy (with or without immunotherapy or antiHER2 therapy),
  3. adjuvant radiotherapy (including local and locoregional radiation treatment with possible radiotherapy boost where indicated).

• patients need to be able to read and understand Croatian.

Exclusion criteria

• female patients with metastatic breast cancer.
• female patients receiving only hormonal treatment.

Trial design

Primary purpose: Supportive Care

Allocation: Randomized

Interventional model: Parallel Assignment

Masking: Triple Blind

---

130 participants in 2 patient groups

Patients Receiving AI-generated Plain Language Summaries

Experimental group

Description:

Experimental Group: participants in this group receive two AI-generated Plain Language Summaries based on the scientific abstract of two Cochrane systematic reviews: Exercise during Adjuvant Therapy (1) and Psychological Interventions for Non-metastatic Breast Cancer (2).

Treatment:

Behavioral: AI-generated PLS

Patients Receiving Original Plain Language Summaries

Active Comparator group

Description:

Active Comparative Group: participants in this group receive two original Plain Language Summaries based on the scientific abstract of two Cochrane systematic reviews: Exercise during Adjuvant Therapy (1) and Psychological Interventions for Non-metastatic Breast Cancer (2).

Treatment:

Behavioral: Original PLS