Pharmaceutical and biotech manufacturers using artificial intelligence (AI) to support compliance with Food and Drug Administration (FDA) regulations should take note: the lack of human oversight of AI quality tools can constitute a current good manufacturing practice (CGMP) violation.
Statements made by the acting director of FDA’s Center for Drug Evaluation and Research (CDER), Michael Davis, M.D., Ph.D., recently underscored this point. As reported by Inside Health Policy, Davis explained:
"AI can be a legitimate aid in document creation and CGMP activities, but any output or recommendation from an AI agent must be reviewed and cleared and authorized by an expert human representative of the firm’s quality unit, before it can be acted upon…. Over-reliance on AI—treating it as a substitute for human expert judgment, regulatory knowledge, or quality oversight, is not acceptable, and will be treated as a [drug CGMP] violation.”
The statements echo an April warning letter to a drug manufacturer which allegedly told FDA inspectors that it “used AI to create drug product specifications, procedures, and master production or control records to be in compliance with FDA requirements.” As FDA explained in the letter, “if you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with [CGMPs]. Your failure to do so is a violation of 21 CFR 211.22(c).”
But the most consequential statement by FDA in the letter was that “any output or recommendations from an AI agent must be reviewed and cleared by an authorized human representative of your firm’s [quality unit] in accordance with section 501(a)(2)(B) [adulteration provisions of the Food, Drug, and Cosmetic Act (FD&C Act)].” Though the manufacturer and its human agents are ultimately liable for violations, this statement begs the question: how much review is actually required?
The Requirement to Review AI Outputs
The warning letter’s presentation of the facts suggests in this case there was an extreme overreliance on AI—with software being used not only to assist in operations but also to inform what FDA’s requirements were and how they could be satisfied (i.e., AI acting as a lawyer). However, FDA’s broad statement that “any output” requires review (and clearance) relates directly to the efficiencies AI can provide. If the level of review for FDA compliance rests upon a “reasonable basis” standard where amount of review scales inversely with the validated reliability of the AI tool (i.e., more reliability equals less review), the standard may drive innovation. However, without clarity on this point, and on what constitutes acceptable validation for given uses, quality unit personnel will clearly have a disincentive to minimize review where they could be held responsible for CGMP violations.
In February 2026, CDER released a list of FDA-announced new and revised draft guidances to be released this year, some of which might help. But the value of AI guidance will depend upon the level of detail it will provide on human responsibilities, what validation FDA views as adequate for systems, and also whether FDA will accept a more active role for itself in reviewing AI manufacturing tools.
Though unlikely based on its current posture toward AI and deregulation, FDA could attempt to directly regulate AI-based products that are promoted for use in meeting CGMP requirements. Historically, there have been very rare instances where FDA approved products used to assure CGMP compliance—most notably Limulus Amebocyte Lysate (LAL) reagents used in bacterial endotoxin testing for drug and biotech manufacturing. Though this scenario raises potential regulatory questions (as a tool used in manufacturing generally is not viewed as a drug or device itself that would be subject to regulation),[1] it is not an impossibility.
More likely, however, is a voluntary program of some kind where FDA performs some assessment of tools. FDA has developed programs like this in the past, such as those regarding drug development tool (DDT) qualification. One could envision that approval requirements or voluntary qualification programs might support greater adoption of AI, be more effective and efficient than manufacturer-by-manufacturer self-validation, and afford developers the opportunity to create regulatory “moats” that protect their investment in developing better AI technologies.
Takeaways
As in many industries, overreliance on AI in the drug manufacturing industry can lead to problems. With or without AI, drugs must be manufactured in accordance with CGMPs and with the Food, Drug, and Cosmetic Act. Liability still runs to the manufacturer.
FDA’s statements are also a reminder to companies that are creating CGMP tools for the pharma and biotech space that they need to take care in how they are representing what their tools can do to help insulate themselves from civil liabilities. Even if not directly regulated, given the increasing awareness that users of AI tend to misuse or over-rely on its output, a regulatory action against a manufacturer could swiftly be accompanied by a civil action where a manufacturer customer seeks damages by alleging that a developer failed to meet its duty of care, mispresented product capabilities, or is otherwise subject to product liability claims.
Finally, the activities FDA may undertake to facilitate AI tool use could be on the horizon, through informal pronouncements, letters to manufacturers, and development of new guidance. It is important for stakeholders to be closely monitoring these activities to consider their implications and advocate for positions of interest.
If you have questions about this article, please reach out to the author.
Epstein Becker Green Staff Attorney Ann W. Parks contributed to the preparation of this post.
Endnotes
[1] FDA, in a response to a request designation (for regulatory classification of a particular endotoxin testing product) stated that it “determined that the product did not require premarket submission to CBER or CDRH because it was not intended to qualify blood or blood products and it was not intended for use in man, animals, clinical diagnosis or patient management.” See FDA PERSPECTIVE ON RECOMBINANT ENDOTOXIN DETECTION SYSTEMS, Slide 31. Nonetheless, there are still BLA-licensed, marketed LAL products, and FDA does have broad authorities it might be able to rely upon to assert jurisdiction.
Blog Editors
Authors
Member of the Firm