Data Science Initiative Talk: David Vidmar - "How Should We Evaluate AI in Medicine?"

About this event

As part of the Data Science Initiative Speaker Series, Dave Vidmar (Tempus AI) presents "How Should We Evaluate AI in Medicine?". Abstract: Technology has long played a significant role in the evolution of medicine, with advancements like electrocardiograms and pacemakers extending what physicians can observe and how they can deliver care. With artificial intelligence (AI) systems becoming increasingly capable, it is natural to wonder what role, if any, such technology should play in the practice of medicine. Proposed applications delegate varying degrees of autonomy to AI, making the question of robust evaluation vitally important. In this talk, we discuss how AI systems in general, and clinical AI systems in particular, have traditionally been evaluated. We examine the unique challenges of applying these conventional approaches to assess performance in messy real-world care settings. We conclude by surveying emerging approaches that aim to provide complementary evidence to evaluate clinical AI systems. Light refreshments will be served before the talk.