Conversations With AI Chatbots Increase Short-Term Vaccine Intentions But Do Not Outperform Standard Public Health Messaging

University of Pennsylvania

Status: Completed

Conditions: HPV Vaccination Intent

Treatments: Behavioral: Public health informational materials; Behavioral: Conversational-style LLM chatbot; Behavioral: Default-style LLM chatbot

Study type: Interventional

Funder types: Other

Identifiers

NCT07132125

LLM_857299

Details and patient eligibility

About

The goal of this clinical trial is to test whether short conversations with large language model (LLM) chatbots can persuade vaccine-hesitant parents to vaccinate their children against human papillomavirus (HPV). The study compares two chatbot styles to official public health information and to a no-message control. Parents of HPV-eligible children first complete a survey about their attitudes toward the HPV vaccine and their main concerns. They are then randomly assigned to read public health materials, have a three-minute conversation with either a default-style chatbot or a conversational-style chatbot tailored to their concern, or receive no message. The main outcome is change in intent to vaccinate immediately after the intervention, with follow-up surveys at 15 and 45 days.

Enrollment: 930 patients

Sex: All

Ages: 18+ years old

Volunteers: Accepts Healthy Volunteers

Inclusion criteria

• Adults aged 18 years and up
• Living in Canada, the UK, or the USA
• At least one child eligible for HPV vaccine based on country-specific guidelines who has not received any doses of the vaccine or whose vaccination status is unknown

Exclusion criteria

• Failure to pass attention checks
• Identified as non-human respondents by reCAPTCHA or other automated bot detection

Trial design

Primary purpose: Prevention

Allocation: Randomized

Interventional model: Parallel Assignment

Masking: None (Open label)

---

930 participants in 4 patient groups

Default-style LLM chatbot

Experimental group

Description:

A large language model chatbot (GPT-4o) that delivers tailored persuasive messages to parents about HPV vaccination, using the model's default style with longer, structured responses. Participants engage in a three-minute, multi-turn conversation, with responses personalized to their top-rated vaccine concern identified in a pre-intervention survey.

Treatment:

Behavioral: Default-style LLM chatbot

Conversational-style LLM chatbot

Experimental group

Description:

A large language model chatbot (GPT-4o) that delivers tailored persuasive messages to parents about HPV vaccination in a conversational style, with short responses. Participants engage in a three-minute, multi-turn conversation, with responses personalized to their top-rated vaccine concern identified in a pre-intervention survey.

Treatment:

Behavioral: Conversational-style LLM chatbot

Public health informational materials

Active Comparator group

Description:

Official public health informational materials on HPV vaccination from the Centers for Disease Control and Prevention (U.S.), National Health Service (U.K.), or Public Health Agency of Canada, matched to the participant's country of residence. Materials are 589-680 words in length, cover HPV risks and benefits of vaccination, and are presented for three minutes before participants proceed.

Treatment:

Behavioral: Public health informational materials

No Message

No Intervention group

Description:

No Intervention