“Companies are pushing hard to incorporate AI into practice,” said Diana Miglioretti, co-lead of the US Breast Cancer Surveillance Consortium, who will lead the trial’s data coordinating center at UC Davis Health. But most evidence for the tools is based on “artificial settings,” she said, where results don’t impact women directly.
So the two-year trial, supported by the Patient-Centered Outcomes Research Institute, will seek to answer critical questions about whether AI deployed in the US results in meaningful changes in breast cancer detection and patient callback rates in the real world. Approximately 400,000 mammograms will be randomized to be assessed by a radiologist alone, or with the aid of an AI-based, FDA-cleared tool from ScreenPoint Medical called Transpara.
“This is one of the first trials that can help us measure the clinical impact of AI solutions when deployed at scale,” said Sanjay Aneja, a radiation oncologist at Yale Cancer Center who studies mammography AI. “We don’t really know if proposed solutions can actually improve the lives of patients or are just part of a growing AI fad that has permeated a number of industries.”
The last time breast imagers adopted a new technology en masse — computer-aided detection software, which similarly flagged areas of concern in mammograms without the use of AI — it drove up costs significantly without improving rates of cancer detection, or worse. “What was shocking is our studies found that CAD actually decreased performance,” said Miglioretti.
“Now AI is the new CAD,” she said. While vendors have received FDA authorization for their AI algorithms that highlight concerning findings on mammograms, those clearances can be based on reader studies that have radiologists look at images in a lab setting. “There’s a lot of hope that these are better than CAD,” said Miglioretti. “But we felt that it was really important to do a randomized trial to see if reader studies are actually reflective of what happens in clinical practice.”
The AI screening trial will aim to recruit participants from seven academic medical centers in California, Florida, Massachusetts, Washington, and Wisconsin — a strength, because “we want to make sure these types of solutions truly scale to all types of patients and practice locations,” said Aneja. “Frequently we see solutions work very well in small institutions, but fail to generalize when scaled to larger more diverse populations.”
Called PRISM (for Pragmatic Randomized Trial of Artificial Intelligence for Screening Mammography), the trial will also measure how often the AI misses cancers or results in false positives. “We want to know whether AI helps radiologists find more cancers, or just flags more exams that ultimately turn out to be normal,” said Joann Elmore, a medicine and health policy professor at UCLA and one of the trial’s principal investigators, in a press release.
While the trial is pragmatic and leaves some elements of AI deployment up to each site, the tool’s use may look different than when deployed in practices today. Some radiologists choose to use an AI tool to assist their read of a mammogram only after they’ve taken their own initial impressions. If a mammogram is randomized to get the AI treatment in this study, the first thing a radiologist will see is a one-page AI report showing risk scores and an automatically marked-up image, although they can then choose to toggle off the AI information and ignore it.
“This trial is an important step forward,” said Manisha Bahl, a breast radiologist at Massachusetts General Hospital. When results begin to be published — likely no sooner than 2028 — they will provide US-based data on the impact of AI in practices staffed by specialized breast imaging radiologists.
There are questions the study won’t answer: It focuses on a single AI-based mammogram interpretation tool, despite the fact that several are FDA-cleared and available in the US It also will not answer whether the AI tools can operate autonomously; similar to standard practice today, mammograms that are randomized to be interpreted by AI will still get a read from a radiologist. “One thing that remains unanswered is if we feel these AI systems can truly function independent of physician oversight,” said Aneja.
But the trial will tackle more nuanced questions about whether and how AI can influence care depending on the patient who gets a mammogram and the provider who interprets it. Researchers will track patient demographics and breast cancer risk factors, as well as detailed information about radiologists, including their clinical experience, previous AI exposure, and level of trust in the technology. All of those could have significant impacts not just on patient outcomes, but whether the AI makes radiologists more efficient in their work.
For breast cancer experts, answers about AI’s real-world impacts can’t come soon enough. “I wish they had these results today,” said Miglioretti, “because it’s being disseminated into clinical practice already.”