Duaer

Guides

Clinical Trial Matching with LLMs in 2027: Why Eligibility Data Becomes the First Bottleneck

Drawn from the Duaer documentation site, this article explains why criteria decomposition and data preparation tend to become the first bottlenecks in LLM-based clinical trial matching, and proposes an evidence-first, auditable pilot approach.

Why screening is far harder than reading charts

Every research site running clinical trials knows the screening problem well. A single protocol often has dozens of inclusion and exclusion criteria, and coordinators must check each one against a patient's record. The answers are frequently buried in free-text clinic notes, scanned discharge summaries, and lab reports that use inconsistent units. A large language model looks like an obvious shortcut. It can read a chart, follow the logic, and produce a shortlist in minutes. In practice, many teams find that the model is not the hardest part. The real difficulty is getting the criteria, the data, and the accountability for screening decisions to line up cleanly.

Consider a typical criterion such as 'no anti-PD-1 therapy within the past 12 weeks' or 'creatinine clearance above 50 mL/min measured within 30 days before enrollment.' Each phrase hides several requirements. The model must identify the correct drug, calculate dates correctly, find the right lab value, and decide whether a record that says 'planned but not started' negates a prior exposure. Any one of these steps can fail silently.

The data adds friction too. Medication history may sit in one system, pathology in another, and clinic notes in a third. Time windows depend on the enrollment reference date defined by the protocol. Coordinators already know these traps, which is why manual screening, though slow, is reliable.

The result is that an LLM can look excellent in a demo and still produce unusable output in a real workflow. A false positive wastes a coordinator's time verifying a patient who never qualified. A false negative means a genuine candidate never reaches the investigator. Neither type of error shows up in a single overall accuracy number.

Four mistakes teams commonly make

The first mistake is asking the model to make the eligibility determination directly. When the output is only 'yes' or 'no,' no one can see why the model reached that conclusion, and the result cannot be audited when the protocol is amended. The model should produce an evidence-backed shortlist, not a final verdict.

The second mistake is passing the entire protocol to the model as one long block of text and hoping it will sort things out. Criteria that depend on one another, such as 'age 18 or older unless enrolled in the pediatric cohort,' are easily lost in long context. Teams that skip decomposition often find that similar patients receive inconsistent results.

The third mistake is testing only on hand-picked examples. A clean set of well-written demonstration charts says little about the messy records in a real screening queue. Without a gold standard built from coordinators' past decisions, there is no way to know whether the system is genuinely good or merely fluent.

The fourth mistake is treating data governance as an afterthought. Patient records often cannot leave institutional boundaries, and a workflow that sends charts to an external API may fail compliance review before it reaches any research site. Teams usually discover this late, after the prototype has already shaped everyone's expectations. For a broader view of the regulatory constraints that limit healthcare pilots, see AI Implementation Challenges in Healthcare: Compliance First, which covers the question of sequencing in more depth.

Decompose the criteria before writing prompts

The right starting point is a criteria inventory. For each protocol, break every inclusion and exclusion criterion into atomic conditions, then assign each condition to one of three categories.

The first category covers structured, directly checkable conditions: age, sex, diagnosis codes, and lab thresholds with units. Once units are normalized, deterministic rules should handle these. An LLM should not be asked to compare 1.2 mg/dL with 106 µmol/L, since that is a problem code already solves reliably.

The second category requires reading comprehension: prior treatment history, symptom descriptions, and performance status documented in narrative form. An LLM adds real value here, provided it returns the sentence or text span on which each judgment rests.

The third category requires human judgment: conditions the record cannot answer, such as an investigator's assessment of disease stability or a patient's willingness to participate. The system should flag these and stop automated processing. This division is among the most useful design decisions a team can make. It narrows the space where the model can hallucinate and keeps every other judgment traceable.

Output evidence and the metrics that matter

The output format determines whether coordinators trust the system. A good candidate record should list each criterion, its status (met, not met, unclear, or not documented), the supporting evidence with source and date, and the rule or model step that produced the result. A coordinator should be able to verify a candidate in about two minutes without rereading the full chart.

This also helps control hallucination. If the model cannot point to a text span in the source document, the output should be marked unverified and kept off the shortlist. Automatically checking that each cited span actually appears in the source text is a simple test that catches a large share of fabricated support.

Accuracy alone is misleading. Teams should track a small set of metrics that reflect real workflow. Recall, measured against a gold standard built from past screening decisions, shows whether the system is missing eligible patients. Precision, the share of flagged candidates that coordinators accept, shows how much time the system wastes. The rate of unresolved criteria shows how often the system correctly admits it cannot decide. The evidence verification pass rate shows whether citations hold up when checked. Finally, measure the total time per screened patient, including coordinator review, because a system that is fast but generates a long review queue does not actually save time.

Rerun the gold-standard tests whenever the protocol is amended, the EHR changes its note templates, or a new site joins. Matching performance can drift at exactly these moments, and without a standing test set the drift is easy to miss.

Start with one protocol at one site

A first pilot should be deliberately narrow. Choose a protocol whose criteria are mostly structured, and run it at a single site whose coordinators are willing to label outcomes. Avoid rare-disease protocols with very few cases, since too few examples make evaluation meaningless. Also avoid oncology protocols with dozens of treatment-history rules on day one, because decomposing that logic alone can take months.

Once the pilot runs, the same pattern can extend to a second protocol. Teams that build a solid data foundation first usually scale faster. Before any prompt design begins, Data Readiness for AI: What to Check Before Your First Model is a useful reference that covers data quality issues in detail.

One step you can take this week

Pull ten past screening decisions from a protocol you currently run. For each one, record which criteria the coordinator assessed, which evidence they used, and the final outcome. Then sort those criteria into the three categories described above. You will end up with a gold standard, a criteria inventory, and a clear picture of which parts are rules, which require reading comprehension, and which need human judgment. That is enough to decide whether an LLM pilot is worth running.

If you want to evaluate criteria-level matching without building the workflow from scratch, you can sign up at https://www.duaer.com, configure decomposition, evidence output, and verification steps in Duaer, and test them on your own protocol before committing to a larger build. Signing up is free, so you can judge the fit with your own data.

Using an LLM for clinical trial matching is worth doing when the output is a reviewed, evidence-backed shortlist rather than an automated decision. Sign up at https://www.duaer.com to test this workflow on your own protocol, and treat the first pilot as a measurement exercise rather than a push toward automation.

Tell us about your use case and we will send back a written rollout path for that scenario. Form: https://api.duaer.com/form/c59a907a-3697-4edf-a04d-3ea0fb72fdc8

Questions

Can an LLM replace coordinators' manual screening for clinical trial matching?

No, it should not. A better role is assistive: the model produces an evidence-backed shortlist that coordinators review, and any condition the record cannot answer, such as an investigator's judgment of disease stability, is handled manually.

How should a team choose the first protocol for an LLM clinical trial matching pilot?

Pick a protocol whose criteria are mostly structured, that has enough cases, and whose coordinators are willing to label outcomes. Avoid rare-disease protocols with too few cases and oncology protocols with complex treatment-history logic as a starting point.

Is accuracy enough to evaluate an LLM-based patient screening system?

No. Evaluate it together with recall against a gold standard, coordinator acceptance rate (precision), the rate of unresolved criteria, the evidence verification pass rate, and total time per patient including manual review. Retest after protocol amendments or template changes.