Keeping a Human in the Loop—AI for Medication Management Requires Clinical Safety Checks

Healthcare leaders share real-world examples and admit they’re still waiting for the best use case for AI. 

When Greek philosopher Aristotle coined the phrase, “The whole is greater than the sum of its parts,” he probably wasn’t talking about AI—but he might as well have been.  

“AI plus clinician is greater than AI or clinician alone,” said DrFirst Chief Medical Officer Colin Banas, M.D., M.H.A., during the recent webinar, “From Data to Decisions: The Vital Combination of AI and Human Expertise for Medication Management and Patient Care.” 

Dr. Banas invited colleagues David Wetherhold, M.D., CMIO of Ambulatory Systems at Scripps Health in San Diego, California, and Dana Darger, RPh, Director of Pharmacy at Monument Health’s Rapid City Hospital in Iowa, for a lively conversation. 

A highlight was when they shared their respective health systems’ results from using DrFirst’s clinical-grade AI solutions to pre-populate missing prescription instructions (sigs) and import medication history into Epic workflows—eliminating data gaps and the burden of manual entry. (See how it works by playing the Med Entry Dash game.) 

At Monument, which has been using DrFirst in its Epic electronic health record (EHR) system since 2019, 93% of patients aged 65+ have medication history available and 82% of home medication sigs are being enhanced by clinical-grade AI.  

Scripps has been live on DrFirst since September 2020, with medication history available for 96% of patients over 65 and 85% of home medication sigs being enhanced by clinical-grade AI. Scripps also reported 271,000 keystrokes in the acute care setting saved per month, which is equal to 75 monthly repurposed hours or nine days  

“This is one of those solutions that we have working every day, and it works so well that our physicians don’t even know it’s happening,” Dr. Wetherhold said. “They just are expecting that most of these prescriptions are coming back with discrete sigs already filled out. And they quickly forgot what was happening before we went live with DrFirst.” 

“This has really made a big difference,” he said.  

At Monument Health, Darger said they used DrFirst previously, but when they switched to Epic in 2017, they employed a different medication history solution for a couple of years before realizing they could switch back to DrFirst. “And the interesting thing was, actually it was cheaper to use DrFirst, and we got better way better information,” he said.  

“So it’s four or five years we’ve been back with DrFirst, and we get great data,” Darger said. 

But even with great data, we still need to “keep humans in the loop,” Dr. Banas said. To drive this point home, he shared several articles in a “Ripped From the Headlines” segment, including a cautionary tale about ChatGPT’s limitations in providing accurate drug information and another about a study revealing ChatGPT’s inability to handle complex problems, including prescription review, patient medication education, and adverse drug reaction (ADR) recognition. 

“These articles are the reason why everything gets a human review,” Dr. Wetherhold said.  

Dr. Banas stressed that ChatGPT and the clinical-grade AI that is part of DrFirst’s medication management solution are two very different things.  

“This is an AI engine that is a decade in the making. It is not a large language model,” he said. “This is something that was clinically curated by clinicians and pharmacists and is constantly iterating and improving over the course of each day.” 

“We are benefiting from that every day,” Dr. Wetherhold said. “I can see it happening when I’m in my clinic on Fridays.” 

Scripps has also implemented AI solutions such as ambient listening, which is already saving doctors as much as 40 minutes to an hour a day, Dr. Wetherhold said. 

“For years and years, everything that we did with EMRs just really added more work for the physician,” he said. “And for the first time, I’m starting to see solutions where we’re making things easier for the physician.”

“I used to be very happy when I could save one of my physicians five minutes of efficiency,” Dr. Wetherhold said. “But the fact is at the end of the day, you really don’t feel that five-minute savings. If you save someone 40 minutes, you feel it. You’re thinking, ‘I can see another patient. I can do my in-basket work. I can go home early and actually eat dinner with my family,’ things like that.” 

Another headline shared by Dr. Banas illustrated an important issue: “Where’s the ROI for AI? CIOs struggle to find it.”  

 “The physicians are more likely to stay around and not burn out,” Dr. Wetherhold said. “But how do I put dollars on that?” 

Darger agreed. “The last couple of years have been tough on everybody,” he said. “Like you said, the ROI is not black and white yet.” 

The last headline, from the Wall Street Journal, makes a bold statement—that the AI revolution is already losing steam. 

“This is how every new thing comes about. It comes out. Everyone gets excited. Then you start realizing some of the limitations of it, which is where we are right now,” Dr. Wetherhold said. “It’s pretty standard, it’s just the hype cycle.” 

“There’s always the visionaries and the futurists, the people who really want to engage with anything new,” Darger said. “So you get a blip on the radar screen where everybody’s excited, but the majority of the world is still sort of in that naysayer category.” 

“And I think that’s where we’re at with AI,” he added. “It takes a little while for the realist to sort of understand that ‘this is here, it is going to work.’ And then you sort of start another exponential growth phase.” 

Physicians and pharmacists are no strangers to new things, said Darger, who is nearing retirement.  

He can clearly remember filling out prescription labels on a manual typewriter, which is obsolete today. But when computer adoption ramped up, “about 95% of my colleagues were like, ‘uh-uh, I’m not putting a computer in my pharmacy,’” Darger said. 

The analogy is apt, he said, because a lot of clinicians feel that way about AI. 

“We’re still growing and we’re still learning,” Darger said. “But there’s so much opportunity there we don’t know yet because we haven’t seen the things that it will do and can do.” 

Dr. Wetherhold agreed: “We haven’t even thought of the best use case for AI in healthcare yet.”