Most of an agent's input is a cache hit

Most of an agent's input is a cache hit

In a day of traffic on GLM 5.3 Flash Xploded, about 94 of every 100 input tokens were read from cache. What that does to the bill, and how to keep it that way.

The number

In the 24 hours to 12 November 2026, GLM 5.3 Flash Xploded answered 1,563 requests carrying 168 million input tokens. About 158 million of them, 94%, were read from cache rather than processed again.

Why agents cache so well

Each turn of an agent starts with exactly what the last turn sent: the same system prompt, the same tool definitions, the same history. Only the newest message and tool result are new. The model keeps the work it already did on that opening stretch and reuses it, so only the new tail is processed from scratch, and the reused part is billed at the cached price.

What cached input costs

Each model has its own cached input price, per million tokens:

A 25-turn session averaging 117,905 tokens a request reads 2,947,625 input tokens. With 94% of them cached, the input on GLM 5.3 Flash Xploded costs about $0.61, and $0.88 if none were. On Kimi K3.1 Pro Xploded the same session's input costs about $3.54, and $22.11 without the cache.

How to keep the cache

  • Keep the start of the prompt fixed. The system prompt and tool definitions go first and stay byte for byte the same. A timestamp or a random id near the top makes every request look new.
  • Add to the history, never rewrite it. Editing an earlier turn breaks the cache from that point on.
  • Stay on one model for a session. Each model keeps its own cache, so switching models mid-session starts again from nothing.
  • Keep the session moving. A cache is kept for a while after the last request, not forever, so a session left idle for a long spell pays for the full prompt once when it resumes.

Seeing it in a response

The response's usage reports it. On the OpenAI-compatible endpoint it is usage.prompt_tokens_details.cached_tokens; on the Anthropic-compatible one, usage.cache_read_input_tokens.

More from the blog