Prompt Caching in 2026: Cut LLM Costs, Keep Quality

Prompt caching is the single highest-leverage cost lever in production LLM engineering for 2026: it stores the computed key-value tensors behind a repeated prompt prefix so that the static portion of every request — your tool definitions, system prompt, and reference documents — bills at up to 90% off, with the model producing byte-identical output. No distillation, no quantization, no quality trade-off. Just a structural change to how you order a prompt.

The reason it matters now is scale. Agentic workloads loop the same 20,000-token system prompt across dozens of steps per task; retrieval pipelines re-send the same document corpus on every query; chat products replay an entire conversation history each turn. Every one of those tokens was already paid for once and recomputed from scratch on the next call. Caching reclaims that spend — but only if the cacheable block is genuinely stable, which most teams discover the hard way.

This guide is the cross-provider reference: how the key-value cache mechanism actually works, current Anthropic, OpenAI, and Google pricing with the write premiums and TTL tiers spelled out, a breakeven matrix that shows exactly when caching saves money versus when it costs more, the prompt-ordering rules that move a workload from a 7% to an 84% hit rate, and the worked cost math behind all of it. Every figure is sourced to a provider doc or a published study.

Prompt caching operates on the key-value (KV) cache — the same data structure that makes transformer decoding tractable in the first place. During the prefill phase, every prompt token is processed and the key (K) and value (V) matrix projections from each attention layer are computed and stored. During decode, each newly generated token attends to those cached K/V matrices rather than recomputing the projections for every previous token. For a repeated prefix, that reduces the attention work from O(n²) to O(n).

Provider-level prompt caching extends that mechanism across requests. When a new request shares an identical prefix with a recent one, the provider reuses the already-computed KV state for that prefix instead of running prefill again. You pay a reduced “cache read” rate for those tokens, and time-to-first-token drops because the expensive prefill is skipped. The catch is that the match must be exact — the cache is keyed on the literal token sequence, so a single changed character before the cache boundary breaks the hit.

The three major providers landed on caching at different times and with structurally different pricing. Anthropic launched prompt caching in public beta on August 14, 2024 and reached general availability on December 17, 2024; OpenAI shipped automatic caching on October 1, 2024 (initially a 50% discount, since raised to up to 90% on newer models); and Google introduced explicit context caching at Google I/O in May 2024, then added zero-setup implicit caching for Gemini 2.5 models on May 8, 2025. The table below consolidates the current 2026 figures — no single vendor page has all three side by side.

Two structural differences are worth internalizing. First, Anthropic charges a write premium — you pay more than base input the first time a prefix is cached (1.25× for the 5-minute tier, 2.0× for the 1-hour tier), then 0.10× on every read. OpenAI and Google implicit caching charge no write premium at all; the trade-off is less control over the cache boundary. Second, Google’s explicit caching adds a time-based storage fee ($1.00/MTok/hour for most models, $4.50 for Gemini 3.1 Pro Preview) that the others do not — so explicit caching only pays off when read volume per stored hour is high enough to amortize that storage cost.

Pairing caching with model selection compounds the savings. If you are routing cheap, cacheable steps to a small model and reserving a frontier model for the hard ones, our deep dive on LLM model routing strategies covers the second half of that equation.

The intuition that caching always saves money is wrong. On a write-premium provider like Anthropic, a low hit rate means you keep paying the inflated write cost without enough reads to amortize it. The decision hinges on one number: your cache hit rate — the fraction of requests that land on an already-cached prefix. The matrix below recomputes the net cost of caching as a percentage of the no-caching baseline at each hit rate, for the four most common configurations.

The formula is straightforward. A request either misses (you pay the write multiplier) or hits (you pay the read multiplier), so blended cost relative to base input is (1 − hitRate) × writeMultiplier + hitRate × readMultiplier. For Anthropic 5-minute the multipliers are 1.25× write / 0.10× read; for 1-hour they are 2.0× / 0.10×; for OpenAI and Google implicit there is no write premium so a miss costs 1.0× and a read costs 0.10× (OpenAI) or 0.25× (Google’s 75% implicit discount).

The matrix exposes the trap. Anthropic’s 1-hour tier costs more than not caching at 50% hit rate and below — its 2.0× write premium needs roughly a 67% hit rate just to break even versus 5-minute, and a 50%+ rate to beat the no-cache baseline at all. The 5-minute tier, OpenAI, and Google implicit caching all stay net-positive across the whole range because their write cost is at or below base input. This is why Anthropic’s documented breakeven of about 1.4 reads per cached write applies specifically to the 5-minute tier.

Because cache invalidation is hierarchical — on Anthropic, any change to a content block invalidates that block and every block after it — the entire optimization reduces to a single principle: put the content that never changes first, and the content that changes every request last. Get the order right and a long, expensive prefix caches cleanly; get it wrong and a single volatile token at the top wipes the cache on every call.

On Anthropic specifically, the discipline is stricter than “put dynamic data last.” Tool definitions must remain byte-identical and in the same order across requests; changing tool_choice, thinking parameters, or an image in the system prompt invalidates downstream cache entries. The provider supports up to four cache breakpoints per request with a lookback window of up to 20 content blocks per breakpoint — enough to cache tools, system prompt, and a document corpus as separate stable segments while leaving the query uncached.

The most instructive published case comes from ProjectDiscovery, whose security agent Neo runs 20 to 40-plus LLM steps per task on top of a 20,000-token system prompt. Their initial cache hit rate was a dismal 7% — because the system prompt contained dynamic working memory that mutated as the agent worked, invalidating the entire cacheable prefix on nearly every step. The fix was a single architectural change: move the dynamic working memory out of the system prompt and place it as a user message at the end of the prompt.

That one relocation raised the cache hit rate to 84% overnight and cut overall LLM cost by 59% — peaking at 70% in the final 10 days of the engagement, with 9.8 billion tokens ultimately served from cache. The lesson generalizes: the highest-value caching work is almost never tuning TTLs or breakpoints, it is auditing what dynamic content has crept into the supposedly static prefix.

Time-to-live is the second decision after prompt order. The cache entry expires after its TTL window of inactivity, so the right tier depends entirely on how frequently the same prefix is hit. Pick wrong and you either pay a write premium you cannot amortize, or you let a still-useful cache expire between requests.

One counterintuitive finding deserves emphasis. The academic study “Don’t Break the Cache” (arXiv 2601.06007, 2026) tested caching across 500-plus agent sessions with 10,000-token system prompts and found that caching reduced API costs 41–80% and improved time-to-first-token 13–31% — but that naive full-context caching can paradoxically increase latency rather than reduce it. Strategic control of the cache boundary outperforms caching everything by default. The implication for engineers: do not just wrap the whole context in a cache breakpoint and assume you have won.

Abstract discounts only land when you run them against a real workload. Below are three worked scenarios with every cell recomputed from the provider’s stated per-token prices. These build on the kind of real-world baselines we measured in our analysis of token cost ROI across 50 agency workflows.

Scenario A — Claude Sonnet 4.6, 50K-token system prompt

One hundred requests a day against a 50,000-token system prompt. With no caching, that is 100 × 50,000 × $3.00/MTok = $15.00/day. On the 5-minute tier at a 90% hit rate: 10 cache writes (10 × 50,000 × $3.75/MTok = $1.875) plus 90 cache reads (90 × 50,000 × $0.30/MTok = $1.35) totals $3.225/day — a 78.5% reduction. On the 1-hour tier at a 95% hit rate: 5 writes (5 × 50,000 × $6.00/MTok = $1.50) plus 95 reads (95 × 50,000 × $0.30/MTok = $1.425) totals $2.925/day — an 80.5% reduction.

Scenario B — OpenAI GPT-5.5, agent loop

A 10,000-token system prompt looped 50 steps per run across 200 runs a day. With no caching: 200 × 50 × 10,000 × $5.00/MTok = $500/day. With 24-hour extended caching at a 95% hit rate (no write premium on OpenAI): misses are 200 × 5 × 10,000 × $5.00/MTok = $50, hits are 200 × 45 × 10,000 × $0.50/MTok = $45, totaling $95/day — an 81% reduction.

Two caveats keep these numbers honest. First, every figure is an input-side saving — output token cost is identical with or without caching, so a workload that is output-heavy will see a smaller blended reduction than these input-only percentages suggest. Second, the hit rates assumed here (90–95%) are achievable only on genuinely stable prefixes; plug your measured hit rate into the breakeven matrix above before promising a finance team an 80% cut.

Most caching failures trace back to a handful of avoidable mistakes. Because the cache key is the literal token sequence, even a capitalization or whitespace difference before the boundary breaks the match — and the resulting miss looks identical to a working cache except for the bill. The defensive rules are simple but unforgiving.

A useful operational habit on Anthropic is pre-warming: send a request with max_tokens: 0 against the system prompt before traffic arrives. The API returns immediately with stop_reason: "max_tokens" and populates cache_creation_input_tokens, confirming the cache is warm before the first user shows up. For monitoring, OpenAI surfaces the cached token count as cached_tokens in the response usage field — wire that into your observability so a hit rate regression triggers an alert rather than a surprise invoice. For the deeper memory-management mechanics underneath all of this, our companion piece on KV cache optimization techniques goes a layer below the provider abstraction.

Looking forward, the next frontier is semantic caching that bypasses the model on near-matches rather than exact ones. Approaches like vCache return cached responses for semantically similar prompts under user-defined error-rate guarantees, saving both input and output tokens where prefix caching saves only input. Expect 2026 production stacks to layer exact-match, semantic, and prefix caches together — and the teams that instrument hit rate per layer will be the ones who actually capture the savings the pricing tables promise.