Does AI Make Clinicians More Appropriately Confident? A Randomized Study in Preterm Birth Prediction

• AI prediction (binary) (Behavioral): AI decision support based on cervical ultrasound providing a binary classification (preterm birth before 37 weeks or term birth) in addition to standard clinical information.
• AI risk estimate (%) (Behavioral): AI decision support based on cervical ultrasound providing an estimate of preterm birth risk (%) in addition to standard clinical information.

Primary outcomes

• Clinician diagnostic calibration (accuracy-confidence alignment) after AI exposure. [Immediately after AI exposure during a single questionnaire session (approximately 20 minutes).]: Agreement between post-AI decision correctness (0/1) and post-AI confidence rating (0-10) will be quantified using the Brier score. Confidence will be rescaled to 0-1 and squared differences between confidence and correctness will be averaged across cases to produce a participant-level score. Lower scores indicate better diagnostic calibration. Results will be compared between randomized arms.

Secondary outcomes

• Helpful switch rate and harmful switch rate. [Baseline (pre-AI) and immediately after AI exposure during a single questionnaire session (approximately 20 minutes).]: Proportion of cases with helpful and harmful switches calculated for each participant and compared between study arms. Helpful switch = incorrect pre-AI decision changing to correct post-AI decision. Harmful switch = correct pre-AI decision changing to incorrect post-AI decision.
• Change in decision accuracy, confidence, and diagnostic calibration from pre-AI to post-AI. [Baseline (pre-AI) and immediately after AI exposure during a single questionnaire session (approximately 20 minutes).]: Within-participant change from pre-AI to post-AI in decision accuracy (proportion of correct decisions), confidence rating, and diagnostic calibration. Differences will be compared between randomized arms and stratified by AI correctness.
• Association between self-rated trust in AI and behavioral reliance on AI. [Immediately after AI exposure during a single questionnaire session (approximately 20 minutes).]: Self-rated trust in the AI output will be measured using a numeric rating scale (0-10) after AI exposure for each case. Behavioral reliance will be quantified as the proportion of post-AI decisions concordant with the AI output. The relationship between trust ratings and behavioral reliance, including concordance when the AI is correct and incorrect, will be evaluated at the participant level and compared between randomized arms.
• Follow-up cervical ultrasound planning. [Baseline (pre-AI) and immediately after AI exposure during a single questionnaire session (approximately 20 minutes).]: Proportion of cases in which clinicians plan an additional cervical ultrasound (yes/no), summarized per participant and compared pre-post AI and between randomized arms.

Sex: All, accepts healthy volunteers

Inclusion Criteria:

* Medical doctors currently working in or training within the field of obstetrics and gynecology.
* Experience performing transvaginal cervical ultrasound examinations.

Exclusion Criteria:

\- No prior experience performing transvaginal cervical ultrasound examinations.

• Martin G Tolsgaard, MD, PhD, DMSc, Study chair, Department of Obstetrics and Gynecology, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark