If you're building anything serious with LLMs, you've probably noticed that your token bills add up fast—sometimes shockingly fast. But there's a technique that's been hiding in plain sight across Claude, GPT, and Gemini that can slash those costs by 70 to 90 percent. It's called prompt caching, and understanding it isn't optional anymore—it's the difference between a side project that stays affordable and one that blindsides you with a five-figure API bill.

What Prompt Caching Actually Is

Prompt caching is a mechanism where providers store a hash of your input context—the system prompt, your instructions, any reference material—and reuse it across multiple requests without re-processing those tokens. Instead of paying full price for that 4,000-token system prompt every single API call, you pay a heavily discounted rate on subsequent requests because the provider is pulling from cached computation rather than running inference from scratch. The math here is straightforward: identical or near-identical prefixes get computed once and amortized across all future calls.

Why Your Bill Is Decided at the Tokenizer

The author breaks down how LLMs count tokens and why billing happens at the tokenizer level—a detail most developers gloss over until they're staring at a bill they don't recognize. Different models tokenize differently, which means identical text can produce wildly different token counts depending on what you're running. When you add prompt caching into the mix, suddenly that 4,000-token system prompt you've been including in every single request becomes nearly free after the first call. The cached tokens typically run at a fraction of the cost—sometimes 90 percent less—which explains how teams are reporting those dramatic savings figures.

Which Providers Support It Today

According to the analysis covering Claude, GPT, and Gemini, prompt caching is available across all three major providers as of mid-2026. Each has implemented it slightly differently: Anthropic's implementation for Claude tends to offer some of the most generous cache discounts, while OpenAI and Google have their own pricing tiers for cached versus uncached tokens. The key requirement is that you need consistent prefixes in your prompts—applications like RAG systems with fixed retrieval templates, agent loops with immutable system instructions, or any codebase where you're sending the same context repeatedly will see the biggest gains.

Real Math: Before and After

The author walks through concrete examples showing how a typical production workload might shift. Consider an application making 10,000 API calls per day, each with a 2,000-token system prompt. Without caching, that's 20 million tokens processed daily at standard rates. With effective caching on the prefix layer, you're processing only the variable portions—maybe 500 tokens per call—while the cached 1,500-token prefix runs at cache rates. The arithmetic is brutal in the best way: your per-request cost doesn't just drop, it collapses relative to what you were paying before.

Key Takeaways

  • Prompt caching discounts apply only to consistent prefixes across requests—no benefit for fully dynamic inputs
  • Cache hit rates depend heavily on how you architect your prompts and whether you can structure them with static headers
  • Different providers have different cache expiration policies—understand when your cached tokens expire
  • The technique works best for agentic systems, RAG pipelines, and any application with fixed system instructions

The Bottom Line

Prompt caching isn't a hack or an optimization trick—it's infrastructure-level cost management that should be in every LLM developer's toolkit from day one. If you're not already structuring your prompts to take advantage of cached prefixes, you're essentially lighting money on fire. The providers have built this discount mechanism for a reason: they benefit when you send more requests with consistent context. Smart developers align their architecture with those incentives and pocket the difference.