The Liability Battle Over Medical AI Is Coming

As artificial intelligence becomes capable of acting more independently in health care, hospitals and technology companies are confronting a fundamental question: Who is responsible when something goes wrong?

Brian Anderson, CEO of the Coalition for Health AI, said tensions are emerging between hospitals and the companies developing some of the world’s most advanced AI models as the technology becomes more autonomous.

“There’s a standoff right now going on with a lot of health systems and the frontier labs about who’s going to be responsible and control the agents,” Anderson said at Newsweek’s AI Health Summit in New York.

In most U.S. states, Anderson said, health care providers remain responsible when something goes wrong while they are using an AI tool.

“They are the liable party should something go wrong with a patient,” Anderson said. “And the same stands true in the instance of using AI tools.”

But responsibility becomes less clear when AI agents act more independently and clinicians may not know why the AI took a particular decision.

“The health systems we’re talking to right now are deeply concerned about the use of agents in high-risk use cases,” Anderson said.

Physician groups are also questioning where responsibility should fall. The American Medical Association argues that accountability should align with the party best positioned to understand an AI tool’s risks and prevent or mitigate harm, rather than automatically falling entirely on physicians.

Anderson said hospitals are particularly concerned that AI agents could “go rogue” and act outside their protected systems and expose patients’ private health information.

“When something goes bad, who’s the responsible party?” Anderson said. “Right now it’s the health system.”

Responsibility could begin to shift as AI systems become more autonomous. Anderson said Utah was considering shared liability and pointed to Doctronic, which is testing AI-assisted prescription renewals in the state and has obtained malpractice insurance.

“I think that might be a signal that we might start seeing more and more technology companies appreciating that there is going to be some shared liability in this space, particularly in the more semi-autonomous use cases,” Anderson said.

For hospitals, however, questions over responsibility remain unresolved as AI systems become more independent.

“This is the rub for a lot of the health systems,” Anderson said. “They appreciate that there is real risk.”