GxP-Agent: DAG Workflows for Reliable Clinical Coding

Clinical trial programming is a poor fit for the “ask the model and hope” style of AI automation. An analysis dataset must follow CDISC conventions, preserve subject-level meaning, and remain traceable enough for review or submission. A syntactically correct script can still produce the wrong variables, records, or transformations. That combination makes the work tedious for programmers and unusually unforgiving for language models.

A recent arXiv preprint proposes a different approach with GxP-Agent. The system does not treat dataset generation as one giant prompt. It turns the regulatory workflow into an explicit execution plan, then assigns parts of that plan to specialized agents. The design is less flashy than asking an LLM to autonomously “solve” the entire task, but it is much closer to how dependable software is usually built: break the job into stages, check the output, and stop errors from spreading.

Why one-shot generation struggles

The paper’s baseline results make the problem clear. Across 11 one-shot generation attempts involving five leading models, none produced a valid subject-level analysis dataset. That is not simply a coding failure. Clinical data preparation contains dependencies about domain structure, variable derivation, ordering, and regulatory conventions. Missing one assumption early in the process can invalidate everything downstream, even when the generated code looks plausible.

This is a familiar failure mode for LLM-based developer tools. Models are often good at producing local pieces of code, yet much less reliable when a task requires a long chain of domain-specific decisions. Retrieval can provide documentation, but documentation alone does not necessarily tell an agent which operation must happen before another, or when an intermediate result should be rejected. GxP-Agent’s contribution is to make that control logic part of the system rather than leaving it inside the model’s uncertain reasoning.

Turning regulatory logic into an execution graph

At the center of GxP-Agent is a directed acyclic graph, or DAG. The complete dataset-generation job is divided into 15 domain-specific nodes. Independent worker agents handle those nodes, while the graph defines their dependencies and order. Each worker receives pharmaverse skill context, giving it relevant tooling and domain guidance instead of relying on a generic prompt.

The graph also contains validation gates. An output is checked at the point where it is produced, and a failed check can trigger a conditional retry. That detail matters more than it may sound. In a flat multi-agent setup, one bad intermediate table can be passed to several later agents before anyone notices. A gated DAG treats each stage as a contract: produce the expected result, or do not allow the workflow to continue normally.

For teams building regulated AI systems, this is a pragmatic architectural choice. The graph becomes a readable representation of the process, while the validation records provide evidence about what happened at each stage. It does not remove the need for human review, but it creates more useful review points than a single block of generated code.

What the benchmark actually showed

The authors evaluated the system on CDISC-Bench, a benchmark built from the FDA pilot submission dataset CDISCPilot01. With Claude Sonnet 4.6 as the underlying model, GxP-Agent reached 100% structural matching in all three independent runs. The result covered 49 of 49 variables and 254 records. That is a strong result within this benchmark, although it should not be mistaken for proof that every clinical programming task is solved.

The comparison is still instructive. The strongest retrieval-augmented baseline reached 59.2% structural matching. Single-agent and flat multi-agent approaches both recorded 0%. In other words, the important variable was not simply the number of agents or the presence of retrieved reference material. The topology of the workflow appears to have supplied the missing structure.

The paper also reports an interesting model effect. GPT-4.1 achieved 59.2% when used with the same DAG architecture, but scored 0% under the other tested architectures. That suggests the graph can act as a form of scaffolding. It gives a less capable model a constrained route through the task instead of asking it to reconstruct the entire process from an open-ended instruction.

Evidence beyond one dataset—and the limits

GxP-Agent was also applied to an adverse-event dataset, ADAE, using a nine-node branching DAG. The test involved 55 variables and 1,191 records, and the system reached 100% structural matching on its first attempt. The branching design is relevant because real workflows are rarely just a straight line; different data conditions can require different paths or checks.

Still, the evidence needs to be read carefully. The work is a preprint and has not yet gone through peer review. The benchmark is tied to a single FDA pilot project, and structural matching is only one measure of quality. A dataset can match expected fields and records while still raising questions about maintainability, edge cases, provenance, or suitability for a real submission process. Broader validation across studies, therapeutic areas, programming teams, and production-like review procedures would make the claim more convincing.

For organizations experimenting with regulated LLM workflows, the practical lessons are fairly concrete:

• Model the domain process before selecting the model. Dependencies, validation rules, and retry conditions may deliver more reliability than a larger general-purpose model.
• Give every stage a testable output contract. A validation gate should catch an incorrect intermediate result before it becomes input for later steps.
• Keep human approval in the loop for consequential work. An auditable DAG improves oversight, but it does not replace qualified clinical programmers or regulatory review.

That makes GxP-Agent interesting beyond clinical trials. The same pattern could apply wherever AI must follow an ordered, inspectable process: encode the workflow, delegate bounded tasks, and verify each handoff. The headline result is promising, but the more durable idea is architectural. In compliance-heavy environments, explicit structure may be a better path to dependable automation than asking the model to reason its way through everything at once.