The US FDA recently published an interesting discussion paper, entitled Using Artificial Intelligence & Machine Learning in the Development of Drug and Biological Products.
Here is a link to the PDF: https://www.fda.gov/media/167973/download
The document is not designed to be an official guidance document. Rather, the report is an attempt by the FDA to start asking key questions about the use of artificial intelligence and machine learning (“AI/ML”) across drug development, including clinical trials and manufacturing. Based on our read of the document, three interesting benefits and concerns emerged.
Benefits
Speed and Cost
AI in drug development can save time and money. This much is exceedingly obvious. But there are many other reasons to deploy AI/ML across drug development, such as the identification of novel targets and entire novel drug classes, intelligent repurposing of existing drugs, and more accurate design and execution of Preclinical and Clinical studies.
Hence, beyond the obvious time and cost savings, AI/ML may result in innovations which would simply not be possible without it. As the document correctly notes, any “discoveries” made by AI/ML will still require biological and chemical validation.
AI/ML in Manufacturing
The production of clinical and commercial batches generates massive amounts of data, and AI/ML can clearly play a role here.
We believe that the design and development of new manufacturing processes (especially for the scaling of API manufacturing) can benefit tremendously from AI/ML approaches by leveraging existing data and experiences to forecast potential processes.
We especially see tremendous potential in the design of clinical and commercial scale manufacturing of complex therapies, such as cell and gene therapies.
Post-Marketing Surveillance
The detection and prevention of post-approval issues, such as adverse events, manufacturing errors, and related issues also requires massive amounts of data handling and analysis. AI/ML can be used to proactively process and format these data for submission.
Importantly, there may be approaches whereby some of these challenges can be predicted and prevented entirely, provided the right AI/ML models are developed and trained properly,
Does the FDA have concerns? You bet.
Garbage In, Garbage Out
We are quite familiar with the GIGO concept when it comes to any sort of modeling (especially financial modeling). But GIGO is especially important to keep in mind with AI/ML.
There are several places in the document where the FDA authors raise the concern about the quality and accuracy of training data used in these models, and the corresponding results and decisions made therein. The GIGO concern spans the entire development value chain, from target discovery all the way to commercial manufacturing. The prediction of three dimensional structures of proteins was also highlighted, but not elaborated upon.
Hence, irrespective of the application, for AI/ML to be valuable and useful relies both on the models AND the accuracy of the training data relative to the problem(s) being solved with those models.
Biases in training data
Related to the GIGO problem is the potential for biases to creep into the training data. For example, clinical trial data spanning a limited patient population may not be easily used for studies outside this defined patient population. Conversely, a widely-distributed in silico patient population can make the drawing of conclusions and projections difficult for specific patient subsets.
The report noted the strong potential of AI/ML to amplify errors and preexisting biases in underlying data sources.
Groping in the dark
Reading through the report, you get the sense that the FDA is very interested in understanding how AI/ML can be used across the entire pharmaceutical value chain, but is simultaneously uncertain how to proceed. This is obviously what drove the report, as it ends with a request for feedback from readers.
A number of questions are posed in the final chapter, as the FDA seeks community insights, feedback, and guidance. It is unclear (at least to us) what exactly the FDA is thinking from a guidance and regulatory perspective.
For example, does it really matter from a regulatory perspective if a new chemical entity was “proposed” by AI? Or if a clinical trial design was “proposed” by AI? Where some FDA guidance will likely be needed is in areas such as submissions, pharmacovigilance, and the like.
And, finally, will the FDA be able to keep up with this very rapidly evolving landscape?