FDA AI Regulation: What Evidence Regulators Expect
Most FDA-cleared AI devices reach the market without demographic data, outcome evidence, or a single randomized trial — and that gap is where liability lives.
Blake Aber · Predicate Ventures · 2026
The market is large and the evidence is thin
The FDA has authorized more than 1,600 AI-enabled medical devices for marketing in the United States as of September 2026. That number reads like maturity. The underlying evidence does not.
Among 691 FDA-cleared AI/ML devices, study design went unreported 46.7% of the time. Training sample size was unreported for 53.3%. Demographic representation was unreported for 95.5%. Only 6 devices, or 1.6%, reported randomized clinical trial data. Three reported patient outcomes.
So a product can be cleared, marketed, and deployed at scale without the public record ever showing who it was trained on or whether it improved a single patient's health. That is the environment teams build in when they build AI for regulated care.
Why the pathway shapes the evidence
The reporting gap is not an accident. It follows from how most devices get cleared.
Of the 168 ML-enabled devices the FDA authorized in 2024, 159 (94.6%) went through the 510(k) pathway, and only 9 (5.4%) through De Novo. The 510(k) route asks a sponsor to show that a new device is substantially equivalent to a legally marketed predicate. It does not, by design, demand fresh clinical trials.
Equivalence to a predicate is a comparison to an existing product, not proof of benefit to a patient. For software that learns from data, that comparison can obscure more than it reveals. Two models can share an intended use and diverge sharply on the populations they handle well.
Radiology dominated 2024 clearances at 74.4%, followed by cardiovascular at 6.5% and neurology at 6.0%. Non-US sponsors accounted for 57.7% of clearances. A model trained on foreign populations, cleared by equivalence, and deployed on US patients carries a validation question that the clearance itself does not answer.
What regulators expect, and what courts will ask
Clearance is a floor, not a shield. The FDA's authorization tells you a device may be marketed. It does not tell a jury that your specific deployment was reasonable.
When an AI recommendation contributes to harm, the questions come in a predictable order.
First: did the device perform as validated on a population resembling the patient? Given that 95.5% of cleared devices report no demographic data, a defendant may have nothing in the public record to answer with. That silence becomes the plaintiff's argument.
Second: did the deployer monitor performance after installation? A model that was accurate at clearance can drift as clinical practice, imaging equipment, or patient mix changes. The absence of monitoring is easier to prove than the presence of harm.
Third: who decided, the software or the clinician? Liability tends to follow the party with the last meaningful choice. Systems that automate away human judgment concentrate risk on the deployer.
Build the record you will need
The defensible position is documentary. Keep the evidence you would want to produce if a regulator or a court asked for it two years after deployment.
That means recording which model version made which recommendation, what data it saw, and what the clinician did with it. It means capturing performance metrics against your own patient population, not only the sponsor's claims. It means a written escalation path for cases where the model's confidence is low or its output conflicts with clinical judgment.
None of this is exotic. It is ordinary engineering discipline applied to a setting where the cost of not having it is measured in patients and litigation.
Privacy is a second regulator sitting at the table
The FDA governs safety and effectiveness. It does not govern how you handle the patient data your model runs on. Two frameworks apply at once, and satisfying one does not satisfy the other.
Training data, inference data, and audit logs are all protected health information the moment they touch a patient. A model that logs its inputs for the sake of the liability record described above has created a new store of sensitive data that must be secured, access-controlled, and retained under its own rules.
The tension is real. Good validation practice pushes you to keep more data. Privacy practice pushes you to keep less. The resolution is scope: keep what you can defend keeping, minimize the rest, and document why each decision was made.
Vendors complicate this further. When a non-US sponsor built the model, the data flows and the contractual terms governing them deserve the same scrutiny as the clearance. A model you did not train is a supply chain, and supply chains carry obligations you inherit whether or not you inspected them.
How to deploy where the stakes are personal
The pattern across all of this is the same. The regulatory clearance is the beginning of your responsibility, not the end of it.
Start by reading the clearance critically. Find the intended use, the predicate lineage, and whatever the sponsor did report about study design and population. Where the record is silent, treat the silence as a question you must answer yourself before deployment.
Validate against your own population before you trust the vendor's numbers. If the model was cleared by equivalence rather than trial, the burden of showing it works on your patients falls on you.
Instrument the deployment so that the record exists before you need it. Version, log, and monitor. Assume that one day someone will ask what happened and why, and build so that you can answer.
Keep a human in the decision where the harm would be personal. The last meaningful choice is where the law looks, and it should be a place where judgment lives.
The volume of cleared AI devices will keep rising. The evidence supporting each one will remain uneven. Teams that treat clearance as permission to stop thinking will find that the thin public record becomes their thin defense. Teams that build their own evidence will be ready for the questions that clearance was never designed to answer.