AI Governance in Clinical Research: What Sponsors Need to Know

FDA's Center for Drug Evaluation and Research reviewed over 500 drug submissions containing AI components from 2016 to 2023, informing the first-ever draft guidance on AI use in regulatory decision-making for drugs and biologics, which the agency issued in January 2025 ^[1]. In September 2024, EMA published a finalized reflection paper placing clinical trial sponsors explicitly among the responsible parties for AI deployed in the medicinal product lifecycle ^[2]. And on January 14, 2026, FDA and EMA jointly published ten guiding principles for Good AI Practice across the medicines lifecycle ^[3].

The regulatory ambiguity that characterized AI deployment in clinical research for the better part of a decade is resolving. What is replacing it is not a single binding rulebook, but something arguably more demanding: a layered mix of binding requirements, nonbinding agency expectations, and international harmonization efforts that sponsors are expected to understand and build into their operations now, before the full enforcement apparatus around AI in clinical research is complete.

This article maps the current governance environment for sponsors deploying AI in clinical trials, covering what FDA, EMA, ICH, and the EU AI Act each expect, where accountability sits when something goes wrong, and what practical governance infrastructure actually needs to look like.

Why AI Governance Now Matters for Sponsors

The phrase "AI governance" sounds like a compliance afterthought. In clinical research, it is the opposite.

A 2025 global survey conducted by Tufts CSDD in collaboration with the Drug Information Association, drawing on 302 respondents from 79 pharmaceutical, biotechnology, and CRO organizations, found that only 10.7% of companies had fully implemented AI across their clinical trial activities ^[4]. Another 30.3% were in early piloting stages. The gap between AI experimentation and AI governance infrastructure is the central operational risk sponsors face right now.

FDA has already demonstrated it will enforce when algorithmic systems in clinical trials fail. In August 2024, FDA issued a warning letter to a clinical investigator and their research center, citing failure to ensure the study was conducted in accordance with the investigational plan under 21 CFR 312.60 ^[5]. A 15-year-old patient received approximately 10 times the intended maximum daily dose of the investigational drug, and the inspection findings pointed to inadequate safeguards in the dosing system used in the study. The case illustrates something important: existing GCP frameworks, investigator obligations, and software validation requirements apply to algorithmic and electronic systems used in clinical research regardless of whether AI-specific regulatory guidance exists. Sponsors who deploy AI tools in trial operations without documented safeguards, validation, and human oversight controls are operating under those same frameworks today.

The Regulatory Picture: What Each Framework Expects

Because no single binding AI regulation currently governs all aspects of clinical trial operations, sponsors face an interlocking set of expectations from multiple overlapping sources. Each has a different legal character, a different geographic scope, and a different focus. Understanding what each one actually requires, and what it does not, is the starting point for building a coherent governance program.

FDA Draft Guidance (January 2025): The 7-Step Credibility Framework

FDA's January 2025 draft guidance, "Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products" [docket FDA-2024-D-4689], is draft and currently nonbinding. It provides recommendations, not requirements, but represents FDA's clearest statement of what the agency expects from sponsors using AI to support regulatory submissions for drugs and biologics ^[1],[6].

The guidance introduces a seven-step, risk-based credibility assessment framework. Sponsors are expected to define the regulatory question the AI model is intended to address, specify the model's context of use (COU), assess model risk based on the influence of AI output on the decision and the consequence of that decision, develop a credibility assessment plan, execute the plan, document results, and maintain the model across its lifecycle ^[6]. The depth of evidence required scales with risk: an AI system that autonomously determines patient eligibility in a pivotal trial requires substantially more rigorous validation than one that flags data anomalies for human review.

The guidance is scoped to AI used to generate data or information for regulatory decision-making, covering safety, effectiveness, and quality. It does not directly govern AI used solely for operational efficiency functions that have no bearing on those three domains ^[6]. FDA's April 2026 Request for Information for the AI-Enabled Optimization of Early-Phase Clinical Trials Pilot Program signals that the agency is exploring AI-enabled trial conduct through trustworthiness concepts aligned with the NIST AI Risk Management Framework ^[15], covering validity, safety, accountability, explainability, privacy, and fairness ^[7]. The pilot, coordinated through the Office of the Commissioner, offers an early indication of where FDA's expectations for AI in trial execution are heading.

FDA's framework supports early engagement where appropriate. The credibility assessment plan is designed to be presented to FDA during pre-submission meetings, giving sponsors the opportunity to align on validation approach before committing resources to a method that may not satisfy agency review ^[6].

EMA Reflection Paper (September 2024): Sponsor Responsibility Is Explicit

EMA's finalized reflection paper on the use of AI in the medicinal product lifecycle explicitly names clinical trial sponsors among the responsible parties for AI systems deployed during the trial, alongside marketing authorization applicants and holders ^[2]. This is a meaningful shift from how accountability was previously understood, and it attaches governance expectations upstream of regulatory submission, into the trial itself.

The paper, while not a binding regulation, establishes agency expectations and is intended to align with the EU AI Act and the EMA-HMA AI Workplan for 2023-2028 ^[2]. EMA introduced two distinct risk categories to replace the earlier broad term "high risk": "high patient risk," for AI affecting patient safety, and "high regulatory impact," for AI substantially influencing regulatory decisions ^[2]. This dual-track approach is designed to avoid confusion with the EU AI Act's separate risk classification system, which uses different criteria.

EMA expects sponsors to consider and systematically manage relevant AI risks from early development onward, with the depth of scrutiny proportional to the context of use and the degree of influence the AI system exerts on study outcomes ^[2]. AI and ML systems used in clinical trials are expected to meet applicable requirements in ICH E6 ^[2].

The FDA-EMA Joint Principles (January 2026): Where Both Agencies Are Heading

The joint principles published by FDA and EMA on January 14, 2026, "Guiding Principles of Good AI Practice in Drug Development," represent the most significant step toward global regulatory harmonization on AI to date ^[3]. These ten principles span the full medicines lifecycle, from early research and clinical trials through manufacturing and post-market surveillance.

The principles are not currently binding. Both agencies have stated they are intended to inform future AI-specific guidance in each jurisdiction, meaning the concepts they contain, including human-centric design, proportional validation, rigorous data governance, multidisciplinary oversight, and lifecycle monitoring, will likely shape future guidance and requirements in both jurisdictions ^[3]. As Jon Walsh of Unlearn.AI described the regulatory direction in 2025: "Sponsors need to clearly specify how a model will be used in drug development, identify potential risks and impacts to the study, quantify those risks, and have a plan to address them by evaluating the model under defined conditions" ^[8]. That framing describes what both agencies are expecting, whether or not a regulation formally requires it today.

ICH E6(R3): Where GCP Data Governance Requirements Already Apply

ICH E6(R3), finalized at Step 4 on January 6, 2025, and adopted by EMA CHMP in December 2024, introduces expanded data governance requirements that directly bear on AI-enabled trial operations ^[9]. Unlike the FDA draft guidance and the joint principles, ICH E6(R3) carries regulatory force in each jurisdiction through its adoption into national and regional frameworks: it is not a standalone global statute, but its GCP requirements become enforceable through the regulatory systems of adopting jurisdictions, including EMA in the EU as of July 2025 and FDA in the United States once a compliance date is formally announced.

The revised guideline adds a dedicated Section 4 on data governance, covering data lifecycle management, security, and system validation. Sponsors remain accountable for delegated activities even when performed by CROs or technology vendors ^[9]. That accountability is particularly consequential for AI deployments: if a vendor-supplied AI system produces erroneous outputs affecting trial data integrity, the sponsor cannot attribute the failure to the vendor alone. E6(R3) places sharper focus on governance, contracts, and ongoing oversight of third parties ^[10]. The validation burden for AI-enabled systems, including electronic data capture platforms, remote monitoring tools, and algorithmic patient-screening workflows, falls squarely within this framework.

For EU sponsors, E6(R3) came into effect on July 23, 2025 ^[9]. FDA published E6(R3) in the Federal Register in September 2025; a formal U.S. compliance date has not yet been announced, though the direction is clear and industry preparation cannot responsibly be deferred ^[10].

EU AI Act: A Evolving Timeline, but Mandatory Compliance Approaching

The EU AI Act (Regulation (EU) 2024/1689) entered into force on August 1, 2024 and is being phased in progressively ^[11]. The Act classifies AI systems by risk level. High-risk AI used in clinical and medical settings, including AI-enabled medical devices and AI systems directly affecting patient safety decisions, must comply with transparency, data governance, human oversight, and conformity assessment requirements ^[11].

Obligations for general-purpose AI models applied from August 2, 2025. The timeline for high-risk AI systems has been updated following the EU Digital Omnibus on AI: the European Parliament and the Council of the EU reached a provisional political agreement on May 7, 2026, to extend high-risk compliance deadlines. Under that agreement, standalone high-risk AI systems listed in Annex III would need to comply by December 2, 2027, and AI embedded in regulated products covered by EU sectoral legislation (including medical devices under MDR and IVDR) would need to comply by August 2, 2028 ^[12],[13]. The co-legislators intend to formally adopt the changes before the original August 2, 2026 deadline for standalone high-risk systems. Sponsors should monitor formal enactment and should not treat the provisional extension as a pause signal.

Applicability to any given AI tool in a clinical trial also depends on what function that tool performs. AI used in clinical management of patients, for diagnosis or informing therapeutic decisions, typically classifies as a medical device under EU law depending on intended purpose and MDR/IVDR qualification, and would therefore fall within Annex I of the AI Act as a product-regulated high-risk system. AI used purely for internal R&D support or operational logistics may not fall within the high-risk classification under the Act. That said, even AI outside the Act's high-risk scope may still require internal governance controls under ICH E6(R3) and applicable GCP requirements if it touches trial data integrity or protocol implementation, and could therefore create inspection risk regardless of its EU AI Act classification. The EU AI Act does not replace EMA's reflection paper or sector-specific guidance; Article 8(2) makes clear that high-risk AI systems embedded in regulated products remain subject to those products' sectoral legislation alongside AI Act obligations ^[11].

A Framework Comparison: What Each Source Expects or Requires of Sponsors

Operational Impact: What Governance Actually Requires From Sponsors

Understanding the regulatory picture is the start. Building the governance infrastructure is the operational challenge. Based on what FDA, EMA, and ICH E6(R3) collectively expect, sponsor AI governance programs need to address at least four domains.

Validation proportional to context of use. The FDA credibility assessment framework ties the depth of validation to the influence the AI model exerts on a decision and the consequences of an error ^[6]. Sponsors need to map each AI application in their trial portfolio to its COU, assess the risk level, and document the validation activities performed. An AI tool used to flag data entry anomalies for human review warrants different validation evidence than one used to stratify patients into treatment arms or generate endpoint adjudication recommendations.

Documentation and audit trails. ICH E6(R3) requires sponsors to implement systems ensuring data reliability and integrity across the trial lifecycle ^[9]. For AI-enabled systems, this means validation before deployment, audit trails capturing model inputs and outputs, and documentation sufficient to reconstruct the decision pathway from AI input to trial conclusion. FDA's draft guidance similarly expects sponsors to document model lifecycle maintenance, including any updates, retraining, or performance changes ^[6].

Human oversight requirements. Across FDA, EMA, and the EU AI Act, human oversight appears consistently as a foundational requirement rather than a discretionary design choice ^{[2],[3],[6],[11]}. Under existing GCP frameworks and ICH E6(R3), investigator and sponsor accountability for trial conduct is not transferable to an AI system or its vendor ^[9]. For any AI system influencing patient safety or regulatory submissions, the governance program must define who reviews AI outputs, under what conditions, with what authority to override, and how those reviews are documented.

Vendor oversight and contractual accountability. ICH E6(R3) establishes that outsourcing a task does not transfer the sponsor's accountability for it ^[9]. Sponsors contracting with technology vendors for AI-enabled services need governance-ready contracts specifying validation requirements, audit rights, performance monitoring obligations, and protocols for managing model updates or performance drift. A 2024 peer-reviewed study by the Tufts Center for the Study of Drug Development, published in Therapeutic Innovation and Regulatory Science, found that RBQM practices were implemented in an average of 57% of clinical trials, with meaningful variation by company size, indicating that consistent adoption of even established governance frameworks remains a work in progress across the industry ^[14].

Governance Ownership: Who Is Accountable Inside the Sponsor Organization

One structural challenge sponsors consistently underestimate is deciding who owns AI governance internally. The answer matters because the obligations span multiple functions: clinical operations (study design, protocol, COU definition), data science and biostats (model development and validation), regulatory affairs (submission documentation, credibility assessment plan preparation, FDA engagement), quality assurance (SOP coverage, audit trails, deviation handling), and legal and privacy (vendor contracts, HIPAA/GDPR, liability apportionment). In practice, governance ownership often defaults to whoever commissioned the AI tool, leaving regulatory documentation gaps that surface during FDA pre-submission review or inspection. A designated AI governance function, or at minimum a cross-functional AI governance committee, with clear escalation paths for model changes or performance deviations, is the structural response most consistent with what E6(R3) and FDA's draft guidance together require.

Regulatory Documentation Considerations

The governance infrastructure described above produces documentation. That documentation has to be submission-ready.

FDA's credibility assessment framework requires sponsors to compile a credibility assessment plan and the evidence generated from executing it ^[6]. For AI models that inform regulatory submissions, this package needs to establish: what the model was designed to do, what data it was trained and validated on, what performance criteria were defined, what the validation results showed, and how the model will be monitored going forward. FDA has encouraged sponsors to present the credibility assessment plan during pre-submission meetings to align on adequacy before the formal submission ^[6].

To make this concrete: consider an AI system used to assist endpoint adjudication in a cardiovascular outcomes trial. A credibility assessment plan for that system would need to define the COU precisely (for example, flagging candidate events for human adjudication committee review, not making final adjudication determinations), document the training data provenance and the similarity between training data and the study population, specify pre-defined performance metrics such as sensitivity and specificity against a historical reference set, document the validation results against those metrics, and describe the ongoing monitoring plan for any performance drift as the study progresses. That last element matters because model performance can shift when the patient population changes mid-trial due to protocol amendments or enrollment expansion.

For sponsors operating in the EU, EMA's reflection paper expects AI systems used in clinical trials to be documented in ways that address both "high patient risk" and "high regulatory impact" assessments, as applicable ^[2]. Expected documentation includes evidence of data governance processes for training and validation datasets, performance monitoring plans, and records of human oversight mechanisms.

FDA's CDER AI Council, established in 2024, provides governance oversight and coordination for CDER's AI-related activities ^[1]. Sponsors using AI in pivotal or high-stakes contexts should consider early FDA interaction through formal meeting requests to align on validation approach before submission.

The Limits of AI in Clinical Governance: What Human Oversight Protects Against

The governance frameworks being built around AI in clinical research reflect something regulators have stated consistently: AI systems produce errors that differ in character from human errors, and those errors can propagate silently before detection.

This is why the FDA credibility assessment framework places the model's context of use at the center of the risk determination. The consequences of failure are context-specific. An AI flagging scheduling inefficiencies carries a low consequence of error. An AI-generated patient prognostic score that influences dose selection carries a high one. For any AI system operating in a high-consequence context, the governance program needs to ensure that outputs are interpretable by qualified reviewers, not just accurate in aggregate on a held-out validation dataset.

The bias and fairness dimension adds another layer. AI trained predominantly on data from specific demographic groups may perform poorly for underrepresented populations. FDA's April 2026 RFI explicitly asked sponsors to address approaches for assessing fairness across demographic and clinical subgroups as part of AI trustworthiness evaluation ^[7]. This is both an equity concern and a regulatory one, and governance programs that ignore it are incomplete by the agency's own stated standards.

Sponsor Action Checklist: Where to Start

The frameworks above translate into a concrete set of preparation steps, regardless of which AI applications a sponsor is currently deploying:

How Kitsa Supports AI-Governed Clinical Operations

The governance expectations and obligations described throughout this article generate a significant documentation workload alongside the conventional clinical records required for regulatory submissions: credibility assessment plans, validation evidence packages, lifecycle monitoring records, audit trails for AI-assisted decisions, and cross-referenced sponsor oversight documentation. KScribe, Kitsa's regulatory document generation platform at kitsa.ai/regulatory-document-generation, supports this through structured authorship records, cross-document consistency checks, and version-controlled audit trails. A practical example: a sponsor using KScribe to generate a protocol and its downstream ICF, DSUR, and CSR, as explored in Kitsa's article on how protocol amendments cascade across clinical trial documents, maintains a traceable record showing which document sections were AI-assisted, what human review occurred, and how content remained consistent across the dossier, the kind of traceability that governance-aware regulatory reviewers are beginning to probe as AI documentation expectations develop.

Key Takeaways

• •FDA's January 2025 draft guidance (FDA-2024-D-4689) establishes a seven-step, risk-based credibility assessment framework for AI models used in regulatory submissions, with validation depth tied to model influence on the decision and consequence of error. The guidance is currently nonbinding but represents FDA's clearest statement of expectations for sponsors.
• •EMA's September 2024 reflection paper explicitly names clinical trial sponsors as responsible parties for AI deployed in the medicinal product lifecycle, not just marketing authorization holders, and introduces "high patient risk" and "high regulatory impact" as the two risk categories for AI governance purposes.
• •The joint FDA-EMA principles published January 14, 2026, set ten guiding principles for Good AI Practice covering human-centric design, proportional validation, data governance, multidisciplinary oversight, and lifecycle monitoring. They are nonbinding but both agencies have stated they are intended to inform future AI-specific guidance in each jurisdiction.
• •ICH E6(R3), finalized in January 2025 and carrying GCP compliance weight, adds a data governance section requiring sponsors to validate AI-enabled systems, maintain audit trails, and retain accountability for delegated tasks even when performed by vendors.
• •The EU AI Act's GPAI model obligations have applied since August 2025. A provisional agreement reached May 7, 2026, under the EU Digital Omnibus initiative would extend high-risk AI compliance deadlines: standalone Annex III systems to December 2027, and AI embedded in regulated products (including medical devices) to August 2028, pending formal enactment ^[12],[13].
• •A 2025 global Tufts CSDD survey of 302 organizations found only 10.7% had fully implemented AI across clinical development activities ^[4], while a separate 2024 peer-reviewed Tufts CSDD study found RBQM practices present in only 57% of trials on average ^[14], illustrating that governance infrastructure consistently lags deployment intent across the industry.
• •Human oversight is a consistent expectation or requirement across these frameworks, with binding force varying by legal status and use case. The AI governance program must specify who reviews AI outputs, under what conditions, with what override authority, and how those reviews are documented.

Frequently Asked Questions

References

You Might Also Like