Feature | Breast Imaging | January 16, 2024 | By Christine Book

RSNA 2023 Sets the Stage with Top Leaders in AI for Breast Imaging

An overview of AI’s history, opportunities and challenges, with the latest clinical trials and regulatory news

Setting the stage for the immense amount of discussion and insights on artificial intelligence (AI) during the 109th Radiological Society of North America Scientific Sessions and Annual Meeting (RSNA23), one of the first sessions brought high-level human intelligence to a crowded room of attendees, including myself, eager to learn the latest.

Presenting on “AI in Breast Imaging” the first day of RSNA 2023 were three highly regarded leaders. Connie Lehman, MD, PhD, FACR, FSBI, of Massachusetts General Hospital, a Professor of Radiology at Harvard Medical School, was joined by Etta D. Pisano, MD, FACR, Adjunct Professor, University of Pennsylvania Department of Radiology, and American College of Radiology (ACR) Division of Cancer Prevention/Chief Research Officer, along with Fredrik Strand, MSc, MD, PhD, Karolinska Institute, Stockholm. A summary of the program is offered here.

The agenda focused on AI and mammography interpretation, and covered three key areas: “Clinical implementation: Our Challenges and Our Opportunities,” presented by Lehman; “Scientific Research Updates: Outcomes in Screening Trials,” presented by Strand; and “Regulatory Issues and Looking Forward: Pathway to Approval, Post Market Testing,” offered by Pisano. A robust Q&A session
and panel discussion followed.

Challenges and Opportunities

In identifying the challenges ahead in the growing application of AI in breast imaging, Lehman put forth, “We are still early in clinical implementation.” She posed key questions for consideration. Will reader studies or simulations translate to community practice? Will results from European studies translate globally? To the latter question, she noted that clinical settings vary across the globe (e.g. double reading vs. single reading, consensus vs. independent readings, super-specialists vs. generalists, recall rate of 2-5% vs. 10-15%). Additional challenges come as the financial model is still unclear, said Lehman. She noted the radiology community will be addressing whether value will be derived from time savings if AI replaces a second reader, and if reimbursement will be offered if assistance to a single reader. From a patient standpoint, the question is whether “out of pocket” payment will be reasonable, acceptable and/or sustainable.

Lehman also offered a historical review of the evolution of artificial intelligence, with a nod to the pioneers on this front, as well as investments and research, from the 1950s to present day. She referenced the work of Shakeel Raja, of TechGenies, with “AI History: The First Summer and Winter of AI.” Milestone moments included: Alan Turing in the 1950s introducing the Turing test in his paper, “Computing Machinery and Intelligence;” the Darmouth Conference in the summer of 1956, which launched the field, and coined the term artificial intelligence; inflated hype and new hopes in the 1970s and 1980s; the explosive growth which was considered “the 4th industrial revolution” with a data explosion that led to improved features and learning in the early 2000s; the birth of deep learning through AlexNet and ImageNet in 2012; the “AI Versus MD” article by Siddhartha Mukherjee, MD, in the Annals of Medicine in 2017; the publication, in 2023, of “The Worlds I See: Curiosity, Exploration and Discovery at the Dawn of AI,” by Dr. Fei Fei Li, Co-Director of the Stanford University Institute for Human-Centered Artificial Intelligence (HAI); from 2020 on, the growing concerns surrounding implications of AI and regulatory trends, with more AI solutions being granted approval by the U.S. Food and Drug Administration (FDA), and emergence of AI applications, as on display in the RSNA AI Showcase.

Lehman closed by offering two significant takeaways. First, said Lehman, “In the AI revolution, we are poised to change our paradigm of how we screen, leverage the strength of global partnerships, knowledge, experience to implement new strategies for how we screen.” She continued, “We will capture the promise of AI for our collective patients when we engage in thoughtful and rigorous clinical science to assess the AI models available and support thoughtful and effective solutions for regulation and oversight.”

Scientific Research Trials, Outcomes

In a highly anticipated presentation, Fredrik Strand, MD, Karolinska Institute, shared findings and insights from three research studies. The first, published in The Lancet Oncology (August 2023), by Lang et al, focused on the MASA Triage randomized trial with vendor Screenpoint; a second, from The Lancet Digital Health (September 2023), from Dembrower, et al, which Strand co-authored, with others, utilizing ScreenTrust CAD Consortium as a key investigator. “Artificial Intelligence for breast cancer detection in screening mammography in Sweden: a prospective, population-based, paired-reader, non-inferiority study” used Lunit’s platform; and the third, published in the journal Nature Medicine (October 2023), applying Kheiron, by Ng et al, “Prospective Implementation of AI-assisted screen reading to improve early detection of breast cancer.”

In summarizing the collective learnings from these most current and important findings, Strand offered key insights relating to the focus on more efficient implementations than current double reading. He noted AI standalone without radiologists appears to offer much less workload and same detection. AI standalone and one radiologist led to less workload and more detection. And, finally, he added, AI triaging and informing equated to less workload and more detection (p=0.052).

He closed by offering three key considerations in preparing for implementation: Validate safety of non-studied combination of AI and mammography equipment; calibrate AI score threshold to achieve target metrics in local set-up; and, score distributions vary by AI and mammography equipment combination.

Updates on U.S Regulatory Pathway for Breast AI

For her part, Pisano, noting that all views expressed were her own and did not represent official positions of ACR, offered that ACR is working with many AI vendors to help them prove the safety and efficacy of their products, and also working to help radiologists understand how AI products may be useful in their unique practice settings. She continued with a comprehensive summary of the current regulation of AI and machine learning (ML) software and intended uses, which are: CADt for Triage; CADe for detection/localization; CADx for diagnosis/characterization; CADe/x for both detection and diagnosis/classification; and CADa/o for acquisition/optimization.

Summarizing the limitations of both standalone performance testing and reader studies, Pisano identified significant questions she feels need to be asked and addressed. Were training and testing sets representative of the population the algorithm will be applied to? What sorts of data were missing from the training and testing? Were readers representative of the radiologists who will use the product? In addressing the limitations of clinical trials, the gold standard, Pisano spoke to the amount of time and expense, and asked the audience to consider whether the patient population enrolled, as well as the sites where they are run, are truly representative of the United States population and clinical sites.

In addressing future directions, she discussed whether real world evidence will serve as a pathway to the use of autonomous AI in breast screening. Pisano offered suggestions moving forward. One was installing AI algorithms that have reached certain performance levels through standalone performance tests in clinics to be tested in clinics before U.S. Food and Drug Administration (FDA) authorization, so as to learn how radiologists actually perform when they are used. Next, allowing patients to opt out of the use of the AI at clinic check-in and, for those who opt in, provide the standard of care interpretation by a single reader. In making final points on the current regulatory landscape, Pisano reinforced that the EU and UK are ahead of the US in implementing and testing AI for clinical practice. Lastly, crediting Fiona Gilbert of University of Cambridge, she reported: The UK National Health Service is going to launch a Real World Evidence AI trial in the next year that will likely be using a design very similar to the one she presented to test the use of autonomous AI as a second reader in their national breast cancer screening program.

During the Q&A session and insightful panel discussion, Lehman enthusiastically answered an early career radiologist who asked for advice, saying: “Welcome to a great subspecialty, your impact will be extraordinary.”

Explore the RSNA Photo Gallery

Find more RSNA news and video s