FHIR + LLMs: Enabling Intelligent Patient Recruitment

Introduction

A cross-sectional study of 2,542 randomized controlled trials registered on ClinicalTrials.gov found that approximately 1 in 5 trials completed within their planned timeframe, the median delay was 12.2 months, and fewer than half met their prespecified enrollment target ^[1]. Not because the science was wrong or sites were poorly managed. Because finding patients who genuinely match a protocol's inclusion and exclusion criteria, buried inside fragmented EHR systems spread across dozens of sites, takes longer and costs more than sponsors can sustain.

This is not specific to surgical trials. A 2020 peer-reviewed systematic review and meta-analysis in the Journal of Medical Internet Research notes, citing prior estimates, that around 80% of clinical trials fail to meet their initial enrollment target and timeline ^[2]. Tufts Center for the Study of Drug Development data show that 11% of investigative sites fail to enroll a single patient and 37% under-enroll relative to their target ^[3], generating per-patient costs at low-performing sites that exceed $130,000 against a benchmark of roughly $14,000 at productive sites ^[3].

Two developments are now converging to address this structurally. The first is the widespread availability of standardized, machine-readable patient data through HL7 FHIR (Fast Healthcare Interoperability Resources). The second is the demonstrated ability of large language models to parse complex natural language eligibility criteria and reason against unstructured clinical records with accuracy levels approaching that of trained human reviewers. Together, they create the conditions for more scalable, data-aware patient recruitment.

This article examines what that combination looks like technically, where published evidence currently places the performance bar, what regulatory requirements and governance constraints apply, and how organizations building clinical research infrastructure should be thinking about the architecture.

Why Recruitment Fails at the Data Layer

Most recruitment bottlenecks are not coordination failures. They are information failures.

Trial eligibility criteria are written in free-form natural language and registered as plain text on ClinicalTrials.gov. A single Phase III oncology protocol may contain 30 or more individual criteria spanning diagnosis codes, prior treatment history, lab value thresholds, genomic markers, and temporal exclusions. Matching those criteria against a patient's record requires reading discharge summaries, lab trends, medication histories, and pathology reports simultaneously, across systems that were not designed to talk to each other.

A 2024 PRISMA-based scoping review in Drug Discovery Today examined 88 publications on NLP approaches to eligibility criteria parsing published between 2000 and 2024 ^[4]. A consistent finding: criteria are often expressed with semantic ambiguity, temporal conditions, and negation logic that rule-based query systems cannot reliably capture ^[4]. The practical result is that manual pre-screening by research coordinators dominates most sites' workflows, consuming hours per patient and scaling poorly as protocol complexity increases.

Even when sites use electronic queries against structured EHR fields, the coverage is incomplete. A 2024 paper in Drug Discovery Today noted that the lack of high-quality standardized datasets and the slow adoption of generative LLMs in eligibility parsing are the field's two most pressing technical gaps ^[4]. Critical efficacy endpoints and nuanced eligibility conditions are frequently embedded in free text: radiology reports, clinical notes, consultation letters. Structured database queries cannot reach them.

This is the specific gap that FHIR and LLMs are positioned to close.

What FHIR Actually Provides for Recruitment

FHIR R4 is the currently dominant version of the HL7 FHIR standard, and its adoption as a mandatory interoperability layer is no longer discretionary for certified EHR vendors in the United States. The 21st Century Cures Act Interoperability Final Rule, published by ONC in May 2020, required certified health information technology to support SMART/HL7 FHIR APIs giving access to patient-level electronic health records ^[5]. The ONC HTI-1 Final Rule, effective February 2024, further updated certification criteria and standards requirements under that framework ^[6].

What FHIR provides practically is a set of structured resource types (Patient, Condition, Observation, MedicationRequest, Procedure, DiagnosticReport, and others) that expose clinical data in a standardized JSON or XML format through REST APIs. The SMART on FHIR protocol adds OAuth 2.0 authorization on top of these APIs, enabling third-party clinical research applications to launch within an EHR session and access a patient's structured data with the appropriate permissions ^[7].

The Bulk FHIR Data Access API extends this to population-level queries. A study published in JAMIA in 2024, based on testing at five healthcare sites between April and September 2023, found that certified EHR platforms exported patient populations at speeds ranging from 1,555 to over 12,000 FHIR resources per minute, with one health information exchange producing over 141 million resources in a single export ^[8]. That throughput means that population-level pre-screening (scanning thousands of patients against a trial's core criteria) is no longer a days-long data extraction project in principle. In practice, export speeds vary considerably by EHR vendor, and not every site has optimized or fully tested its Bulk FHIR implementation for production research workloads ^[8].

An ASTP/ONC data brief tracking hospital adoption of patient engagement capabilities between 2021 and 2024 found that FHIR-enabled patient access in outpatient settings grew from 49% in 2021 to 64% in 2024 ^[9]. The infrastructure is increasingly in place. The question is what sits on top of it.

Where LLMs Enter the Matching Pipeline

FHIR provides structured clinical data. It does not solve the natural language problem. Eligibility criteria remain free-text documents. Clinical notes, discharge summaries, and radiology reports contain critical eligibility-relevant information that does not map cleanly to discrete FHIR resource fields. The structured data available through FHIR is necessary but not sufficient.

This is precisely where LLMs contribute. Their ability to read unstructured clinical text, extract named entities, reason over temporal sequences, and evaluate logical conditions, including negations and value thresholds, maps directly onto the tasks that human reviewers perform during pre-screening.

A 2024 peer-reviewed study published in PubMed ^[10] evaluated GPT-3.5 Turbo and GPT-4 against the n2c2 2018 cohort selection dataset, using 202 longitudinal patient records and 13 selection criteria. GPT-3.5 Turbo achieved an accuracy of 0.81 and micro F1 of 0.79; GPT-4 achieved accuracy of 0.87 and micro F1 of 0.86 ^[10]. The study used a retrieval-augmented generation approach, embedding clinical notes into a vector database and supplying relevant text passages for each criterion evaluation.

A separate proof-of-concept preprint study (not yet peer-reviewed) in surgical oncology evaluated GPT-4o across nine pancreatic cancer trials using 24 patients ^[11]. The model correctly identified 39 out of 42 eligible patient-trial matches (90.7%) and agreed with human raters on 2,438 out of 2,520 individual binary criteria (96.7%) ^[11]. Median per-patient screening cost was $0.67, with a median processing time of 138 seconds ^[11]. For context, the human labor to perform that same review would typically be measured in hours per patient at a site.

TrialGPT, a three-stage pipeline published in Nature Communications in 2024 ^[12], decomposed the matching problem into candidate retrieval, criterion-level assessment, and trial-level aggregation. Evaluated on 1,015 patient-criterion pairs, it achieved a criterion-level matching accuracy of 87.3% with explanations for each decision ^[12].

The scoping review published in PMC in 2025 covering 24 studies on LLM-based patient-trial matching found that 21 of the 24 papers were published in 2024, reflecting the rapid pace of development in the field ^[13]. Most systems used patient-centric matching architectures, with OpenAI's GPT family as the most commonly tested model, though open-source options including Llama 2 and Mistral were increasingly evaluated ^[13].

An important caveat applies to this evidence base. The 2025 PMC scoping review noted that fewer than 5% of LLM biomedical studies used real patient data ^[13]. Most published evaluations, including the studies discussed here, use retrospective benchmark datasets, synthetic patient cohorts, or small clinical data samples. Performance figures derived from these settings may not transfer directly to deployment against live, longitudinal EHR data from multiple institutions with variable documentation practices.

The Two-Layer Architecture

What the published implementations share is a two-layer architecture. The first layer uses FHIR APIs to pull structured patient data (demographics, active conditions, lab results, medication lists) and applies rule-based filters to identify a candidate cohort relevant to the trial's primary indication. This narrows the population rapidly without requiring LLM inference on every patient record.

The second layer applies LLM-based reasoning to the candidate cohort, reading clinical notes and unstructured records for each patient and evaluating them against the full set of eligibility criteria. The JMIR AI framework analysis of LLM-enabled recruitment systems noted that strong FHIR interoperability reduces the extraction and mapping burden substantially before LLM-enabled matching begins, and that weak interoperability can undermine the reliability of the entire pipeline ^[14]. The quality of the FHIR data layer is not incidental to performance; it is foundational to it.

A cloud-native recruitment system published in ScienceDirect in 2024, called recruIT, demonstrated this layered approach using FHIR as the interoperability standard and the OMOP Common Data Model as the patient data repository, generating automatic recruitment recommendations by continuously querying the patient population against pre-loaded eligibility criteria ^[15]. Usability evaluations by study personnel showed that the system was deployable in real research environments, though the authors noted that data freshness depends on the frequency of the FHIR-to-OMOP synchronization pipeline ^[15].

What FHIR Structured Data Can and Cannot Cover

The practical performance of any FHIR-plus-LLM recruitment system depends heavily on which eligibility conditions can be evaluated from structured FHIR resources and which require inference over unstructured text.

Demographic criteria, current medication lists, coded diagnoses, and standard laboratory values are generally available through FHIR Condition, MedicationRequest, and Observation resources under the US Core Implementation Guide, though completeness and coding consistency vary across EHR vendors and institutions. These criteria can often be evaluated with structured FHIR queries where the underlying data is present and standardly coded.

More complex criteria present gaps. Genomic eligibility conditions (KRAS mutation status, HER2 amplification, microsatellite instability) may not be available as structured FHIR data at most sites. A 2025 proof-of-concept study published in Studies in Health Technology and Informatics evaluated FHIR Genomics resources combined with Clinical Quality Language (CQL) to encode molecular eligibility criteria, finding that even complex molecular criteria can be represented given proper FHIR specification conformance, though the study acknowledged that most institutions do not yet have genomic data in FHIR-structured form ^[16].

Qualitative imaging findings present a similar challenge. ONC's USCDI standard, which defines what data elements certified EHRs must support through FHIR APIs, mandates access to imaging narrative reports through DocumentReference resources but does not require the exposure of discrete quantitative imaging measurements as structured data ^[6]. This means values such as fat fraction percentage from MRI-PDFF or specific tumor measurements are typically available only in free-text radiology reports at most institutions. FDA's 2024 guidance on real-world data from EHR sources addresses the handling of unstructured data in regulatory submissions: it expects sponsors using tools such as AI or NLP to document and validate those methods, and to maintain traceability from the source text through to the study database ^[17]. While that guidance targets data used in regulatory decisions rather than pre-screening specifically, its validation and traceability expectations are informative for any LLM-based extraction system that may generate data entering the study record. This is a meaningful constraint for trials in hepatology, oncology, and cardiology where critical criteria are captured only in narrative imaging reports.

The practical implication is that FHIR structured data handles a substantial portion of pre-screening efficiently. LLMs provide the necessary complement for criteria requiring inference over clinical notes and free text. Recruitment architectures that rely exclusively on either are leaving performance on the table.

Structured FHIR queries handle the upper portion of this table efficiently. LLM-based reasoning over unstructured text is the primary mechanism for the lower categories.

Regulatory and Governance Considerations

Using FHIR-connected EHR data for clinical trial pre-screening involves regulatory considerations that go beyond pure technical implementation.

Under the HIPAA Privacy Rule, covered entities (healthcare providers, health plans, and health care clearinghouses) and their business associates must obtain patient authorization or an IRB/Privacy Board waiver before using or disclosing protected health information for research recruitment purposes ^[18]. Sponsors typically are not themselves covered entities under HIPAA, but they operate through covered entities (institutions and sites) that are ^[18]. The practical result is that bulk FHIR queries against a health system's patient population for pre-screening require formal authorization from the institution: an IRB-approved protocol, a data use agreement, and in many cases a waiver of authorization, not merely technical API credentials.

FDA's 2024 guidance on real-world data from electronic health records sets expectations around data provenance for EHR-derived data used in regulatory submissions, requiring that sponsors maintain an audit trail documenting the source, extraction, and transformation of data from EHR through to study databases ^[17]. While that guidance addresses data used in regulatory decisions rather than pre-screening specifically, it signals FDA's general expectation of traceability for EHR-derived evidence in clinical development.

The 2026 JMIR AI analysis of LLM recruitment systems identified privacy constraints, governance costs, and audit requirements as factors that materially shape deployment outcomes in real organizational settings, even when the technical matching performance is strong ^[14]. The authors noted that systems performing well under controlled technical conditions may require substantial additional infrastructure (consent management, audit logging, de-identification pipelines, and human oversight checkpoints) before they can operate reliably in institutional research environments ^[14].

Human oversight is not optional. A 2025 PMC scoping review noted that less than 5% of LLM biomedical studies used real patient data, and that fairness, bias, uncertainty, and deployment considerations are rarely measured in published evaluations ^[13]. Systematic bias in LLM-based screening, for example underperformance on patients with rare conditions, atypical clinical presentations, or records concentrated in non-English clinical notes, can affect enrollment equity in ways that aggregate accuracy metrics do not capture.

For oncology trials, the CodeX and Vulcan FHIR Accelerator workstreams have been developing clinical trial matching standards to support consistent patient identification across institutions ^[19]. The HL7 mCODE standard, developed through CodeX, provides a standardized oncology data model that, when implemented in FHIR, gives LLM-based systems substantially more consistent structured data to reason against ^[19].

AI and Automation Perspective

The evidence, taken together, supports cautious optimism about what FHIR-plus-LLM architectures can achieve in patient recruitment, alongside clear-eyed realism about where they fall short.

On accuracy, GPT-4 class models have consistently demonstrated criterion-level matching in the 87 to 97 percent range across multiple published evaluations ^{[11],[12],[13]}. That performance is not equivalent to a trained clinical reviewer. Misclassified criteria in published studies include complex temporal conditions, negation-within-negation logic, and criteria that require integrating information distributed across multiple records from different time points. Sensitivity and specificity vary by trial, by therapeutic area, and by the quality of the underlying FHIR data.

On workflow integration, the most productive published systems treat LLM outputs as decision support rather than autonomous gatekeeping. The 2026 JMIR AI framework describes this explicitly: LLM-generated eligibility assessments serve to triage the candidate pool and surface potential matches for human review, rather than to make final enrollment decisions ^[14]. This mirrors how similar AI-assisted workflows have been implemented in imaging and pathology: the model flags, the clinician decides.

Retrieval-augmented generation has emerged as a consistent architecture in high-performing systems. By embedding patient records in a vector database and retrieving the most relevant text passages for each criterion evaluation, RAG approaches avoid context window limitations and focus the model's reasoning on the specific clinical content most relevant to each eligibility check ^[10].

Fine-tuned models have not consistently outperformed well-prompted general-purpose LLMs in this domain. A 2024 analysis noted that fine-tuning generative LLMs on biomedical domain data showed limited relative performance gains in criterion evaluation tasks ^[13]. The clinical trial eligibility matching task may rely more on reasoning and contextual integration than on domain-specific vocabulary recall, which favors strong general-purpose models with structured prompting over specialized fine-tuned variants.

How Kitsa Fits Into This Problem

KScreener, Kitsa's patient pre-screening product, is built directly on FHIR-connected EHR workflows. It reads structured patient data from FHIR resources to generate eligibility signals against protocol criteria, supporting site coordinators and research teams in identifying candidate patients before manual chart review begins. Because Kitsa's infrastructure connects across the protocol layer (KScribe) and site network layer, eligibility criteria defined during protocol development can propagate directly into KScreener's matching logic, reducing the re-keying and manual interpretation that typically sits between a finalized protocol and the start of active pre-screening at a site. KScreener outputs are designed as decision support for clinical staff review, not autonomous enrollment decisions, and the workflow is designed to maintain the audit trail required for IRB reporting and GCP compliance. These are Kitsa product design goals; independent third-party validation of KScreener's specific implementation is not cited in this article.

Key Takeaways

• •A cross-sectional analysis of 2,542 surgical randomized controlled trials found fewer than half met their prespecified enrollment targets; across disease areas, 11% of investigative sites fail to enroll a single patient, consistent with recruitment being a structural information access problem rather than a site management failure ^[1],[3].
• •FHIR R4 APIs, mandated for certified EHR vendors under the 21st Century Cures Act, now provide standardized population-level data access that supports automated pre-screening workflows at a scale that was not feasible through earlier EHR integration approaches ^[5],[8].
• •Published LLM evaluations show criterion-level patient matching accuracy between 87% and 97% for GPT-4 class models, using RAG architectures that retrieve relevant clinical text for each eligibility check ^{[11],[12],[13]}.
• •FHIR structured data can cover demographic, diagnostic, laboratory, and medication criteria when data is present and consistently coded; genomic criteria, quantitative imaging values, and nuanced clinical status generally require LLM inference over free-text records ^[16],[17].
• •Real-world deployment requires institutional data governance, IRB oversight, HIPAA-compliant data handling, and human review checkpoints: technical accuracy alone does not constitute a deployable recruitment system ^[14],[18].
• •The most effective published architectures use a two-layer pipeline: FHIR-based structured queries to build a candidate cohort, followed by LLM reasoning over unstructured records for the full eligibility assessment ^[14],[15].
• •Enrollment equity remains an underexamined dimension of LLM-based screening; bias in model performance across demographic groups or non-standard clinical presentations needs evaluation before broad deployment ^[13].

FAQ

References

You Might Also Like