Copyright newsweek
Editor’s note: This is AI Impact, Newsweek’s weekly newsletter where each week, we will explore how business leaders are unlocking real value through artificial intelligence. Tap here to get this newsletter delivered to your inbox. Inference Layer By Gabriel Snyder When I came across a new study by Yale researchers showing the effectiveness of AI ambient scribes in reducing doctor burnout, I was immediately reminded of conversations I had in Sonoma last June at the Newsweek AI Impact Summit with leaders and technologists wrestling with a deceptively simple question: How do you actually measure the ROI of AI in health care, especially when the technology is evolving faster than our ability to track its impact — or even before we fully know what kinds of returns from AI we’re looking for? The promise of AI ambient scribes—tools that listen in on patient visits and automatically generate clinical notes—has been a hot topic in health care for a while. Newsweek health care editor Alexis Kayser explored this issue in a September 2024 cover story, “Is AI the Cure for Doctor Burnout?” and in a panel discussion at our headquarters with health system leaders. The hope is that by having AI do the heavy lifting of documenting patient visits, the ambient scribes will reduce the administrative burden on physicians and their resulting burnout, thus stemming the tide of critical staffing shortages by keeping more doctors practicing medicine. But as many speakers in Sonoma noted, the reality of proving what works in AI often lags behind the pace of innovation. As Dr. Allen Chang of UMass Memorial Health put it, “We owe it to ourselves to really prove convincingly that AI or whatever investments we make ultimately can benefit patient outcomes, and that’s a difficult thing to show.” That’s why the new study, with lead authors Kristine D. Olson and Daniella Meeker of the Yale School of Medicine and published in JAMA Network Open, is so noteworthy. Surveying 263 clinicians across six health systems (the study is “the first large, multicenter evaluation to assess the effect of AI scribes on clinician experience”), it found that after 30 days of using an ambient scribe, the percentage of doctors reporting burnout dropped from 51.9% to 38.8%—a 74% relative reduction. Clinicians also reported less time spent on documentation after hours (so-called “pajama time”), lower cognitive load, more ability to focus on patients and a willingness to see more patients per day. Still, it’s important to note what the study didn’t measure: long-term effects, patient outcomes and perspectives, or effects on doctors beyond self-reported surveys. As Dr. Chang pointed out, this echoes the challenge that ballooning electronic health records (EHR) introduced — the very problem ambient scribes are now being asked to fix. “Prior to the adoption of EHRs, we were on paper record systems. We believed that this is going to show amazing benefits in all sorts of ways, but after a lot of time and a heck of a lot of money, believe it or not, we have not still been able to show that EHRs, clinically, have made tremendous benefits,” Dr. Chang said. “Not because they haven’t, but because it’s difficult to show, and it’s had a lot of unanticipated consequences as well.” So, while this new data is promising, suggesting that the hopes for AI tools may be coming to fruition, whether they help health systems reach their ultimate goal — healing patients — remains unanswered. Upcoming Events AI Impact Awards & Summit The Newsweek AI Impact Awards seek to identify and recognize uniquely innovative AI solutions that solve critical business problems in different industry segments, or significantly advance capabilities. Recognition comes not from ideas, but from measurable IMPACT on business operations. Register now – Early bird deadline ends 12/19. Core Intelligence Paul McDonagh-Smith of MIT Sloan says the “GenAI Divide” reflects how ready organizations are to learn, adapt and reinvent themselves. Paul McDonagh-Smith, senior lecturer at the MIT Sloan School of Management, argued that the reason most AI projects fail has little to do with technology itself and everything to do with the environments into which they’re introduced. As he explained to Newsweek, organizations often treat AI as a project to implement rather than a capability to embed. The result is a proliferation of pilot programs that never scale, not because the models are flawed, but because the foundations around them are. “AI implementation isn’t the same as AI adoption,” McDonagh-Smith emphasized. He points to three failure points that consistently appear: governance, culture and data alignment. When roles and responsibilities for AI aren’t clear, governance breaks down. When teams don’t trust or understand the systems they’re asked to use, culture stalls adoption. And...
