Article Text

Download PDFPDF
287 Automatic evaluation of shared decision making in maternity care with natural language processing
  1. Alex Waddell1,
  2. Trang Vu2,
  3. Wendy Pollock3,
  4. Zerina Lokmic-Tomkins3,
  5. Vanessa Wakins4,
  6. Kathrene D Valentine5,
  7. Patrick Olivier1
  1. 1Department of Human Centred Computing, Monash University, Clayton, Victoria, Australia
  2. 2Department of Data Science and AI, Monash University, Clayton, Victoria, Australia
  3. 3School of Nursing and Midwifery, Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, Victoria, Australia
  4. 4School of Nursing and Midwifery, Centre for Quality and Patient Safety Research in the Institute for Health Transformation, Deakin University, Burwood, Victoria, Australia
  5. 5Department of General Internal Medicine, Massachusetts General Hospital, Boston, USA


Introduction Patients often feel excluded from maternity care decisions.1–3Shared Decision Making (SDM) adoption faces significant difficulties; manual measurement methods are resource-intensive.6 This study aims to develop an Artificial Intelligence Natural Language Processing (NLP) method for in- situ SDM measurement. Automatic evaluation would allow timely and consistent measurement to understand the extend of SDM adoption, provide feedback to clinicians and guide implementation.

Method SDM conversations in simulated patient-midwife interactions were transcribed, then scored using OPTION 5.7 Instances of each OPTION 5 item in transcripts were annotated to train the NLP models. A regression model will be used to predict overall OPTION 5 scores and a detection model to identify instances and nuances of SDM in the conversation. Models are based on pre-trained language models, including Med-BERT8 and ClinicalBERT.9 Data augmentation techniques such as paraphrasing will be applied to training data to overcome data scarcity.

Results 25 midwives completed SDM training and practiced in simulated clinical consultations with an actor. 4-fold cross validation will be conducted and the regression model will be evaluated on the mean square errors of predicted item scores at the conversation level. At the sentence level, we will report accuracy of the event detection model in identifying SDM events in the test dialogues.

Discussion The annotation process highlighted limitations in SDM measurement, lacking granular detail to reliably guide coding of conversational elements. Despite this, the initial results demonstrate the potential of automated SDM measurement models on simulated dialogues. The next phase will focus on validating the models on recordings from actual maternity clinic visits to determine effectiveness in realistic clinical contexts.

Conclusion This study shows the possibility of automatic evaluation of SDM in maternity care conversations. It offers potential actionable feedback for integrating SDM into standard practice.


  1. Yuill C, McCourt C, Cheyne H, Leister N. BMC Pregnancy Childbirth. 2020;20(1):1–21.

  2. Lee N, Kearney L, Shipton E, Hawley G, Winters-Chang P, Kilgour C, et al. Women Birth. 2023;36:(6):e574-e581.

  3. Watkins V, Nagle C, Kent B, Street M, Hutchinson AM. Midwifery. 2022;113:103420–103420.

  4. Tracy MC, Thompsonl R, Muscat DM, et al. Z Evid Fortbild Qual Gesundhwes. 2022;171:5–21.

  5. Barton JL, Kunneman M, Hargraves I, et al. MDM Policy Pract. 2020;5(2):2381468320963781.

  6. Waddell A, Goodwin G, Spassova G, Bragge P. MDM Policy Pract. 2023;8(2):1–12.

  7. Elwyn G, Tsulukidze M, Edwards A, Legare F, Newcombe R. Patient Educ Couns, 2013;93(2):265–271.

  8. Rasmy L, Xiang Y, Xie Z, Tao C, Zhi D, NPJ Digit. Med. 2021;4(86):e39305.

  9. Huang K, Altosaar J. Ranganath, CHIL’20, 2020;1:1–9.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.