Disclosure: we build CodeRouter, a router for coding agents. This post is about a bug in CodeRouter and the data from our own team's traffic before and after we fixed it.
A colleague told us Claude Code felt much more expensive through CodeRouter than it should. Our cache numbers said otherwise: 99% of the input that could come from cache did. Both were true. The hit rate we were looking at leaves out the most expensive part of prompt caching: writing to the cache.
Hit rate hides cache writes
Each Claude request's input tokens fall into three groups: read from cache, written to cache, and plain uncached input. On Claude Sonnet 5.5 the list prices per million tokens are:
| Input token type | Price per million |
|---|---|
| Uncached input | $2.00 |
| Cache write, 5-minute | $2.50 |
| Cache write, 1-hour | $4.00 |
| Cache read | $0.10 |
A one-hour write costs 40× a read. In our traffic, 81–85% of cache writes were one-hour writes.
The common hit-rate formula is reads ÷ (reads + uncached input). Writes appear in neither part. A session that rewrites a large share of its prompt every turn can still show a hit rate near 100%, because the tokens it rewrites are counted nowhere.
What our traffic showed
Our team uses Claude Code through CodeRouter every day. These are Claude-model requests from our own accounts, excluding benchmark runs.
| Before the fix (Nov 7–10) | After the fix (Nov 10–11) | |
|---|---|---|
| Requests | 2,516 | 1,575 |
| Input tokens | 454M | 184M |
| Hit rate: reads ÷ (reads + uncached) | 99.1% | 99.8% |
| Reads ÷ all input tokens | 85.2% | 96.5% |
| Share of input tokens written to cache | 14.0% | 3.3% |
| Cache writes' share of the input bill | 84% | 55% |
| Input cost per million input tokens | $0.629 | $0.222 |
The last row prices every token at Sonnet 5.5 list prices, so a change in which models we used doesn't move it. Input cost per million tokens fell 2.8×. The hit rate barely moved.
In a healthy Claude Code session each turn reads the conversation so far from cache and writes only what is new: the last reply, tool results and the new message. That is the 3.3% after the fix. Before it, 14% of every input token was a write.
The bug
Claude Code sends system reminders as system-role messages in the middle of the conversation. CodeRouter moved them into the top-level system prompt and dropped their cache markers, the handling older Claude models need because they don't accept system messages inside the conversation.
The top-level system prompt comes before the conversation. Changing it changes the start of the prompt, and a prompt cache only matches an exact prefix. So whenever a reminder appeared, the whole conversation after it was written to cache again, and some turns weren't cached at all.
Current Claude models accept those messages where Claude Code puts them. Since November 10 CodeRouter passes them through unchanged on those models. For older models it folds each reminder into the neighbouring user message, in place, and keeps its cache marker. We checked 2,644 pairs of consecutive requests from the same conversations: the new code breaks the cached prefix in none of them, the old code in 586.
What to check in your own setup
- Track write share next to hit rate: cache writes ÷ (uncached + reads + writes). In our Claude Code traffic around 3% is normal. Ten percent or more means something rewrites the start of your prompt.
- Look for anything that changes the start of the prompt per request: a gateway that edits the system prompt, tools in a different order, a timestamp or request id in the system text, or a model switch partway through a session (a different model has its own cache).
- Price writes separately: a cache write costs more than uncached input, and a one-hour write costs 40× a read.
The CodeRouter dashboard now shows write share next to hit rate, per model and for 7 or 30 days.
Method and limits
- Data: CodeRouter's request logs for our own team's accounts, Claude models only, successful requests. Token counts are the usage numbers Anthropic returns with each response.
- Before: Nov 7, 00:00 PT to the fix's deployment at 19:33 PT on Nov 10. After: from then to the evening of Nov 11. The after period is about one working day; workloads differ from day to day.
- Costs are list prices, all priced at Sonnet 5.5 to compare like with like. They are not what we paid.