How Can FDA Regulation Build Transparency into Health Care AI Tools?

In October, the U.S. Food and Drug Administration (FDA) held a workshop on the regulation of artificial intelligence (AI) in health care.

While the FDA carefully reviews clinical trial data in a public process before approving new drugs, the agency reviews data for new AI devices privately and only provides a summary of testing after granting clearance. This can undermine clinicians’ confidence in AI devices, according to Jack Resneck Jr., MD, a dermatologist at the University of California San Francisco and president-elect of the American Medical Association. “Just as we want to know risks and benefits of a new drug, we like to see published peer-reviewed data to develop trust in an AI tool as well,” he said.

Medical device manufacturers and their trade groups argued that devices using AI don’t require special regulation from the FDA. Physicians and researchers attending the workshop disagreed, asserting that AI tools must be carefully tested and monitored, as they tend to be opaque and recycle bias embedded in health data.

“There is nothing unique to AI about issues of accuracy, validity, and completeness of data,” said Ralph Hall, JD, professor of practice at the University of Minnesota Law School who spoke on behalf of the Artificial Intelligence in Imaging Coalition. “To create special automatic rules for AI … may slow patient access to what can be some very important products,” he added.

“We have this really incredible technology that can do all these interesting things,” said Lily Peng, a product manager in Google’s health research division. “But at the end of the day, we need to understand how we study this in real-world settings and how we get reporting back to [the FDA] in real time so that you can address problems even before an adverse event actually happens.”

Agency officials have suggested developers of AI tools could disclose expected updates and modifications before they are available for commercial sale so that the FDA could review those changes without triggering another review process.

Source: STAT, October 18, 2021.