The Same 12 Questions Cost $0.53 in One Chat and $0.18 in Another | Edition 331
Edition 331 — Paste your reference material once and keep the session open, or start fresh each time — same task, a very different bill.
Same reference document. Same twelve questions. Same AI model. One way of asking them costs $0.53. The other costs $0.18 — for identical answers. The difference isn't a smarter prompt or a cheaper plan. It's whether you opened a fresh chat for every question, or asked all twelve in one sitting.
That's not a hypothetical — it's Anthropic's own published pricing for Claude Sonnet 5, run through a worked example below and verified against the vendor's pricing page as of August 2026. And the mechanism behind it — a discount for reusing text the model already saw, and a penalty for making it remember that text in the first place — shows up at every major AI vendor, even though the exact numbers and rules differ enough that assuming one company's terms apply to another will get your math wrong.
How the three rates actually work
Every major AI API prices three kinds of input tokens, not one:
- Fresh input — text the model hasn't seen before in this exchange. Billed at the model's base input rate.
- Cache write — the first time a block of text (a system prompt, a reference document, a long instruction set) is marked for the model to hold onto. This costs more than fresh input, not less — it's a premium for making the model remember something.
- Cache read — every later request that reuses that same held text. Billed at a steep discount off the base rate.
The catch, and the reason this is a habit question rather than a technical one: that held memory doesn't last forever, and on at least one vendor it isn't free to leave sitting there unused. Structure a work session so the cache gets written once and read from repeatedly, and you come out ahead. Structure it so a fresh chat re-writes (or fully re-sends) that same material every time, and you pay full — or premium — price on it, over and over.
Three vendors, three sets of rules — verified as of August 2026
The roughly-90%-off-on-reuse pattern shows up at all three major API vendors, but the mechanism behind it is not identical, and treating one company's caching rules as universal will misprice your own usage:
| Vendor (example model) | Cache write premium | Cache read discount | Default cache lifetime | What breaks it |
|---|---|---|---|---|
| Anthropic — Claude Sonnet 5 | 1.25× base input (5-min cache) or 2× base input (1-hour cache, paid) | 0.1× base input (90% off) | 5 minutes, refreshed each time it's reused; 1-hour option costs more to write | Editing any text at or before the cached section; going longer than the lifetime with no reuse |
| OpenAI — GPT-5.6 | 1.25× base input | 0.1× base input (up to 90% off) | Fixed 30 minutes on GPT-5.6+, refreshed on reuse (older models: 5–60 min, or up to 24h on an extended-retention setting) | Falling below the model's minimum cacheable length; a traffic burst routing your request to a different server than the one holding the cache |
| Google — Gemini 3.1 Pro Preview | None published — Google's own docs describe the initial write as billed like ordinary input, not a separate surcharge | 0.1× base input (90% off) on reuse | Whatever TTL you set — defaults to 1 hour if unset, no minimum or maximum | The TTL running out — and unlike the other two, you're billed a separate per-hour storage fee for as long as you keep the cache alive, used or not |
The worked example
Picture a consultant spending an afternoon working from one 20,000-token reference document — a client's policy binder, roughly 15,000 words — and asking 12 follow-up questions about it over a few hours, using Claude Sonnet 5. Anthropic's published rates as of August 2026: input $2 per million tokens, output $10 per million tokens, a 1-hour cache write at $4 per million tokens (2× input), and a cache read at $0.20 per million tokens (0.1× input). Each question runs about 150 tokens and each answer about 400 tokens.
Style A — a fresh chat for every question. Twelve separate conversations, each one re-pasting the full 20,000-token document alongside that question. Fresh input: 12 × 20,150 tokens = 241,800 tokens at $2/MTok = $0.4836. Output: 12 × 400 = 4,800 tokens at $10/MTok = $0.048. Total: $0.5316.
Style B — one session, document cached. The first question pays a one-time cache-write charge on the document (20,000 tokens × $4/MTok = $0.08), plus its own question and answer. The next eleven questions each read the cached document at the discounted rate (20,000 tokens × $0.20/MTok = $0.004 per question) instead of re-sending it — $0.044 total. Question text across all 12 turns (never cached): $0.0036. Output across all 12 turns: $0.048. Total: $0.1756.
Same 12 questions, same document, same model, same answers. Style A costs just over 3 times what Style B costs. Run that same habit once a workday for a 21-day month and the gap is $11.16 versus $3.69 — about $7.50 a month for one person's one recurring task. Scale it across a team, a bigger document, or more questions per session, and the multiplier holds even as the dollar figure grows.
What to actually do about it
You don't need to touch a cache-control flag or an API parameter to get this right. Most consumer AI apps — Claude.ai, ChatGPT, Gemini — already keep a conversation's earlier turns in context, so the discount applies automatically as long as you stay in one conversation. The habit that matters is entirely yours to change: when you're working from the same reference material — a policy document, a codebase, a long brief — set it up once at the start of a session and keep asking follow-up questions there, instead of opening a new chat for each one. The moment you start a new chat, you're back to paying fresh-input (or cache-write) rates on material the model already knew.
The one thing worth watching: how long you can leave a session idle before the discount resets varies by vendor — anywhere from 5 minutes to an hour by default, longer only if you opt into (and pay more for) an extended cache. Try the interactive below with your own numbers: move the gap slider past a vendor's cache lifetime and watch the savings disappear — that's exactly the kind of lunch-break gap that quietly erases the discount in the middle of a long research session.