Although FDA is actively expanding its use of artificial intelligence with agentic AI systems “designed to achieve specific goals by planning, reasoning, and executing multi-step actions,” the agency is not nearly as open to industry’s use of such systems to comply with FDA regulations. In fact, a company recently received a warning letter for its use of AI to create “specifications, procedures, and master production or control records to be in compliance with FDA requirements.”
While there are many valid uses for AI in manufacturing, as discussed in a previous TAG article, this is not one of them. Rather, the citation shows that FDA is focusing in on AI uses and is setting some strict parameters for the industries in its purview.
Following is a discussion of the key citations in the warning letter and how these apply to food manufacturers:
- “If you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with CGMP.”
FDA is not saying that AI cannot be used to assist in the writing of documents, but the use must have extensive human oversight, review, and validation. In its use of agentic AI, FDA notes that the systems “incorporate built-in guidelines — including human oversight — to ensure reliable outcomes.” Without such human oversight, the accuracy is likely to be faulty, and regulatory compliance questionable – as can be seen in the next citation:
- “Overreliance on artificial intelligence for your drug manufacturing operations was also documented during the inspection.” In response to an inspection observation that was out of compliance with regulations, the facility’s response was that they were not aware of the legal requirement, as the AI agent they used never told them it was required.
Not only is lack of knowledge of a regulation never an excuse, blaming AI for not including information on the requirement simply furthers the observation that the facility does not understand or have control over its processes and the applicable regulations. While regulatory compliance can be complex – whether in drug or food manufacturing, it is imperative that every facility be fully knowledgeable of the agency and regulations to which it is subject, or work with an external food safety consultant who is.
- “Any output or recommendations from an AI agent must be reviewed and cleared by an authorized human representative.”
AI has distinct limitations in that it can only repeat what it has learned from its internet trolling; it does not know your facility, equipment, processes, employees, suppliers, etc., and every facility in the food industry is different – even when producing the same product. It is not feasibly possible for an AI agent to have access to all the information needed to dictate the appropriate and compliant processes of a particular facility, and it is literally impossible for AI to validate its implementation.
Additionally, FSMA requires that facilities have a PCQI (Preventive Controls Qualified Individual) who is responsible for the writing, validation, and implementation of the Food Safety Plan. The very name of “PCQI” references an “individual,” for which the legal definition is “a single person as distinguished from a group or class” – thus a human.
As previously stated, AI has it place and can be a helpful assistant, but it should never lead a process, be relied on for 100% accuracy or knowledge, be used to run your food safety programs. AI’s role in the food industry is evolving fast, and using it in a way that allows AI to do the heavy lifting on data analytics so the humans can focus on risk management, and leverage AI’s data analytics, is certainly a good way to start using the strength of AI. Give TAG a call if we can be your human resource for assistance in your food safety efforts and regulatory compliance!
As AI has evolved, TAG has provided a number of resources on its potential uses and its limits related to the food industry. Following are just a few:
Summary: Although FDA is actively expanding its use of artificial intelligence, it is not nearly as open to industry’s use of such systems to comply with FDA regulations. FDA is focusing in on AI uses and is setting some strict parameters for the industries. Without human oversight, the documentation accuracy is likely to be faulty, and regulatory compliance questionable. AI has its place and can be a helpful assistant, but it should never lead a process or be relied on for 100% accuracy or knowledge.
