Enterprise AI teams keep deferring reproducibility infrastructure because they believe it's expensive. The numbers don't support that belief. Here's a defensible cost model for the runtime audit-trace persistence, environment fingerprinting, and attribution-stability measurement that regulated AI deployments will need over the next three years.

A common pattern in conversations with enterprise AI engineering teams: someone — usually a senior engineer or an MLOps lead — agrees that reproducibility infrastructure for production LLM deployments is becoming necessary, then immediately follows with a deferral. "We'll build it when the regulator forces us. Right now we can't justify the cost."

That deferral rests on an assumption nobody validates. The assumption is that reproducibility infrastructure is expensive. In conversations where I've pressed on actual numbers, the responses range from "I'd guess maybe six figures a year" to "honestly I have no idea, but it sounds like a lot of engineering time." Neither answer is the result of a cost model. They're the result of pattern-matching against the perceived complexity of the words deterministic execution replay, environment fingerprinting, audit-trace persistence, attribution-stability measurement.

When you actually run the numbers, the deferral looks less defensible. The infrastructure costs of running reproducibility-aware production AI deployments at meaningful scale are an order of magnitude lower than most teams imagine. The engineering cost of building this infrastructure once is similarly bounded. The cost of not having it — recreating audit defense in three years under regulatory pressure — is the part most teams underestimate.

This article works through a defensible cost model, then explains why the math has shifted in the last eighteen months.

The four cost categories

Reproducibility infrastructure for production LLM deployments has four cost categories. Treating them separately is the only way to talk about the economics without descending into hand-waving:

1. Audit-trace storage — the durable persistence of the full execution trajectory for each decision the system makes. Includes input, model version, prompt template hash, intermediate states, retrieved documents, attribution outputs, and final response. Stored for the audit retention window — typically five to seven years for regulated industries.

2. Compute overhead for measurement — the additional computation required to measure attribution stability under variance (TreeSHAP with K rotated background samples, calibration estimation under bootstrap), rather than producing single-shot explanations.

3. Engineering build cost — the one-time engineering effort to build the deterministic seed handling, environment fingerprinting, persistence pipelines, and measurement protocols into the production deployment. Plus ongoing maintenance of this infrastructure across model and dependency updates.

4. Operational overhead — the marginal cost of running each prediction through the reproducibility-aware pipeline rather than the naive pipeline. Latency overhead, additional infrastructure complexity, monitoring requirements.

Most teams collapse these into a single intuition ("expensive") without separating which category drives the cost. The honest exercise is to put numbers on each one.

Audit-trace storage: the cheapest part

Start with the part that intuitions overestimate most badly. A typical production LLM agent decision produces an audit trace of roughly 50-200 KB when persisted in full — input prompt, system prompt hash, retrieved context documents, intermediate reasoning steps, final response, attribution outputs, environment fingerprint. Call it 100 KB per decision as a working figure for a moderately complex agent workflow.

A high-volume production deployment runs roughly 10,000 to 100,000 decisions per day. Take the upper bound. At 100,000 decisions × 100 KB = 10 GB per day = 3.65 TB per year of audit-trace data.

Five-year retention on AWS S3 Glacier Deep Archive (the appropriate storage class for audit data accessed only on regulatory challenge) costs approximately $0.00099 per GB-month, or roughly $45 per TB-year. Five years of audit traces at the volume above costs around $820 per year in storage. Seven years — the longer retention window some regulators imply — costs around $1,150 per year.

For a deployment running 100,000 production decisions per day, audit-trace storage is rounding error on the AWS bill. Teams that defer reproducibility infrastructure on storage-cost grounds are deferring against a cost that doesn't exist. The math is not subtle.

The real cost considerations on storage are about retrieval latency and metadata overhead. Glacier Deep Archive has 12-48 hour retrieval times, which is acceptable for regulatory audit cycles but unsuitable for operational analytics. Most teams will want a hybrid: hot storage (S3 Standard at ~$0.023 per GB-month) for the last 30-90 days where operational debugging is likely, with lifecycle transition to Glacier Deep Archive for older records. The math still works out cheap.

Compute overhead for measurement: real but bounded

This is where the genuine cost lives. Reproducibility-aware deployments measure things that single-shot deployments don't. Specifically, attribution stability under variance — instead of computing a single SHAP attribution per decision, you compute K attributions across rotated background samples and report both the mean attribution and its variance.

In the reproducibility protocol for imbalance-aware financial distress prediction in my journal manuscript currently under review at Intelligent Computing, the across-repeat protocol measures attribution stability at K = 50 rotated background samples with 100 observations each. The compute cost of this is roughly 50× the compute cost of a single-shot SHAP attribution.

For TreeSHAP applied to tree-ensemble models (XGBoost, LightGBM, CatBoost), this is genuinely cheap — a single TreeSHAP attribution typically runs in 10-50 milliseconds. 50× of that is 500ms to 2.5 seconds of additional compute per decision. On modern cloud infrastructure, this translates to roughly $0.0001 to $0.0005 in marginal compute cost per decision. At 100,000 decisions per day, that's between $10 and $50 per day of additional compute — $3,600 to $18,000 per year.

For deep neural network attributions (Integrated Gradients, DeepLIFT), the picture is harder. K = 50 multiplier on deep-network forward and backward passes can be expensive enough to require infrastructure planning. Many teams will choose K = 20 or K = 30 in production based on the variance characteristics of their specific model — the IC protocol measured K = 50 because we were characterising the variance, not optimising for production cost.

The honest cost takeaway: measurement compute overhead is the dominant cost of reproducibility-aware deployments, but it sits in the thousands-to-low-tens-of-thousands per year for moderately scaled production systems. Not the six-figure intuition most teams carry.

Engineering build cost: the one that actually matters

The largest and least visible cost is the one-time engineering effort to build the reproducibility infrastructure in the first place. This is where the deferral pattern becomes self-defeating — engineering cost is sunk now (when it's bounded) or incurred later under pressure (when it scales unfavourably).

A defensible estimate: building the four reproducibility primitives (deterministic seed handling, environment fingerprinting, persistent audit traces, measurement protocols) into a production deployment that wasn't designed with them takes one senior engineer roughly three to six months of focused work. Less if the deployment is greenfield, more if it requires retrofitting a complex existing pipeline with embedded state assumptions.

UK senior data engineering compensation in 2026 is verified at a London median of £75,000-82,000 per year base, with fully-loaded cost (employer contributions, equipment, overhead) typically landing around £100,000-110,000 per engineer-year. Six months of focused work is therefore roughly £50,000-55,000 in fully-loaded cost — the high end of the estimate. Three months is around £25,000-30,000.

Set that against the alternative scenario: retrofit reproducibility infrastructure under regulatory pressure in three years. The work is materially harder because the deployment will have accumulated three years of architectural assumptions that assumed reproducibility wasn't required. Realistic estimate: 12-18 months of senior engineering effort, plus opportunity cost of engineering time pulled off other work, plus business risk of operating without audit defense during the rebuild window. Cost: £150,000-300,000+ in pure engineering effort, plus the harder-to-quantify costs of regulatory exposure.

The 6:1 to 10:1 cost ratio between building now and retrofitting later is the engineering-economics argument that should drive the decision. Most teams have never seen the comparison laid out because they've never tried to model the retrofit cost honestly.

Operational overhead: marginal at production scale

The marginal cost of running each prediction through reproducibility-aware infrastructure versus a naive equivalent is small once the infrastructure exists. Persistence I/O to durable storage adds 5-20 ms of latency per decision. Environment fingerprinting at the deployment level is one-time per container build, not per decision. Measurement protocol execution (the K attribution draws) is the dominant operational overhead, and it's the compute cost already accounted for above.

For deployments where end-to-end latency matters operationally — real-time fraud detection, conversational agents — the audit-trace persistence can run asynchronously in a separate worker process, removing it from the critical path entirely. This pattern reduces operational latency overhead to near-zero at the cost of slightly higher engineering complexity.

For deployments where latency is less critical — overnight batch processing, async decision workflows — operational overhead is negligible compared to the model inference cost itself.

The cost picture, consolidated

Putting the four categories together for a moderately scaled production LLM deployment (100,000 decisions per day) gives a defensible cost envelope:

• Storage: £600-900 per year (Glacier Deep Archive at scale)
• Compute measurement: £3,000-15,000 per year (varies with model architecture and K)
• Initial engineering build: £25,000-55,000 (one-time)
• Ongoing operational overhead: marginal, absorbed into existing infrastructure

Annualised over a five-year deployment horizon, the total cost lands in the £10,000-30,000 per year range for a 100,000-decision-per-day production system. As a fraction of the total cost of running such a deployment (which typically includes hundreds of thousands per year in inference compute, model hosting, vector database infrastructure, and operations), reproducibility infrastructure is a 5-15% overhead.

That's not free. But it's also not the six-figure-per-year burden the deferral pattern assumes.

Why the math has shifted

Three trends have made reproducibility infrastructure materially cheaper over the last eighteen months than enterprise AI teams' intuitions reflect:

1. Storage cost compression. AWS S3 Glacier Deep Archive at $0.00099 per GB-month is now an order of magnitude cheaper than the on-premise tape archive systems that older audit-defense practices were designed against. Five-year audit-trace retention used to be a real budget line item; now it's a rounding error.

2. Framework maturity. LangGraph, DSPy, Inngest, and similar orchestration frameworks now ship with checkpoint-enabled state persistence as first-class primitives. The engineering cost of "build deterministic state machine with persistent checkpoints from scratch" used to be most of the build cost; it's now a configuration choice. Teams that started reproducibility work in 2024 paid a different price than teams starting in 2026.

3. Measurement protocol availability. Open-source TreeSHAP implementations, calibration libraries, and statistical-stability measurement code are now standard. The protocols that needed to be designed from research papers in 2022-2024 are now available as configurable libraries. My own reproducibility package on Zenodo (DOI 10.5281/zenodo.20454212) is one of many such artifacts; the broader ecosystem has been compounding.

The cost intuitions that drive deferral were largely formed when these three trends hadn't matured. The intuitions haven't updated. The math has.

The honest economic argument

The strongest version of the economic argument for building reproducibility infrastructure now isn't "it's cheap." It's "it's cheap if you build it now, and the cost ratio inverts sharply if you build it later under regulatory pressure."

Teams that frame the decision as "can we afford the infrastructure now" are asking the wrong question. The right question is "what is the expected cost differential between building now versus retrofitting under pressure, weighted by the probability that regulatory pressure materialises within the audit window?"

Given Gartner's projections that AI governance platform spending will reach over $1 billion by 2030, the PRA's increasing emphasis on supervisory challenge of model decisions, the FCA's emerging AI guidance, the EU AI Act's high-risk system requirements, and the EBA's machine learning guidelines — the probability that some form of audit-defense requirement materialises within a five-year window for regulated industries is high enough that the expected value calculation favours building now.

The cost numbers above make this calculation easier to defend. Building reproducibility infrastructure into a production LLM deployment costs roughly £25,000-55,000 in initial engineering plus £4,000-15,000 per year ongoing. The retrofit alternative costs £150,000-300,000+ in engineering plus accumulated regulatory risk during the rebuild. The break-even probability — the threshold at which deferring becomes more expensive than building — is small enough that risk-neutral decision making clearly favours building.

The deferral pattern persists not because the economics support it but because the cost intuitions are outdated and the regulatory pressure is invisible until it isn't.

What this means for engineering planning

For enterprise AI teams looking at this analysis and wondering whether to commit budget, the practical implication is simple: the infrastructure costs are bounded and manageable, the engineering cost is paid once at well-defined scale, and the cost differential between now and later favours acting decisively in the next twelve months. Specifically:

• Storage cost is not a planning consideration. It's a line item small enough to absorb into existing infrastructure budgets.
• Compute measurement is the dominant ongoing cost. Plan for it explicitly. Choose K values that match your variance characteristics and production budget rather than defaulting to research-grade K = 50.
• Engineering build is the largest single cost. Treat it as a focused six-month project for one senior engineer. Don't try to distribute it across an entire team in 10% slivers — the architecture decisions don't compose well across part-time effort.
• The retrofit cost is the cost most teams have never modelled. Modelling it honestly is the most important step in the budget conversation.

The reproducibility infrastructure that enterprise LLM deployments will need over the next three years is genuinely affordable. The bottleneck isn't the cost; it's the willingness to take the cost seriously enough to plan for it.

The teams that act on this in 2026 will have audit defense capabilities built into their deployments at one-tenth the cost of teams that build them in 2029 under regulatory pressure. That's not a marketing claim. It's an engineering-economics calculation with verified numbers. The question is whether the calculation gets done before or after the regulator asks.