The rapidly advancing capabilities of Artificial Intelligence (AI) systems are set to be transformative, particularly in healthcare. AI’s promise in various medical domains is evident, especially in Shared Decision Making (SDM) enabled and analyzed through Large Language Models (LLMs). LLMs, trained on vast text corpora, adeptly represent word and sentence meanings, making them ideal for analyzing extensive conversation data.

Our symposium’s first segment presents a systematic review of LLMs in comprehending turn- based dialogue. We examined their strengths, limitations, training methods, general applicability, and performance in dialogue comprehension, offering a comprehensive understanding of LLM capabilities to evaluate conversations.

We then report on our use of AI to analyze patient-clinician SDM conversations. Our work applied the psycholinguistic theory of conversational alignment which posits that conversational partners unconsciously align their communication patterns to achieve mutual goals and used BERT, an LLM, to complete textual analyses of these conversations. After discussing our findings, we will review the potential of expanding our textual AI model into a multidimensional model by which speech, textual and visual conversational data are analyzed together to evaluate SDM in clinical conversations.

The symposium concludes by addressing the ethical implications and potential risks associated with using LLMs in SDM research. We will present the results of interviewing SDM researchers to understand their views on ethical challenges and potential barriers in using AI for SDM research.

This comprehensive analysis of the viability of using AI to evaluate the textual, aural, and visual dimensions of clinical conversations highlights its potential to revolutionize SDM analysis and to improve the use of SDM in the care of patients.