Cache Point Design

The Goal Stated Plainly

A long-running agent’s cost factors into two multipliers: token count × price per token. These are two levers on the same axis — compaction shrinks the token count; caching drops the already-sent prefix to the read price. This page is only about the price lever.

The biggest number on the price lever is the cache hit ratio. At Anthropic’s pricing a hit costs 10% of an uncached read, a write costs 125% (5-minute TTL) or 200% (1-hour TTL). Every input token lands in exactly one of three buckets, so the expected price is their weighted average:

C = 0.10 × p(hit) + 1.25 × p(write, 5m) + 1.00 × p(uncached)

where C is the mean price per input token (relative to the uncached price, 1.0×), the shares sum to p(hit) + p(write) + p(uncached) = 1, and a 1-hour TTL swaps the write coefficient 1.25 → 2.00. Output is billed separately and isn’t cached. On Sonnet 4.6 ($/M tokens): input $3.00, cache read $0.30 (0.1×), cache write $3.75 (5m, 1.25×) / $6.00 (1h, 2×), output $15.00. A 30-step task where the stable prefix is re-sent each step, written once and read on the other 29: per MTok of prefix, 30 × $3.00 = $90 uncached drops to $3.75 + 29 × $0.30 = $12.45 — about one-seventh. So the engineering goal is single: move p(hit) as close to 1 as possible, and hold it there as the conversation grows and compaction fires.

Anthropic-only: manually placing cache breakpoints is an Anthropic-specific requirement. OpenAI, Gemini, Qwen, and most other providers cache prefixes automatically server-side and neither accept nor need cache_control; for them the breakpoint mechanism below is a no-op (createCachedInstructions / markPrefixCacheBoundary return their input unchanged for non-Anthropic models).

How a Hit Happens

A breakpoint (cache_control) hashes the entire prefix from the start of the request through its block and looks that hash up:

the exact prefix was written before → read it from cache at 0.1×;
not found → cold-write a new entry (1.0×–1.25×) for a later request to hit.

So the cache matches a whole prefix, not a single block — which gives the one invariant: change any earlier byte and every later breakpoint’s prefix hash changes, voiding all of them. For the lookup to hit, two things must hold at once:

Byte-identical: every byte from the request start to the breakpoint matches the prior entry exactly.
Within reach: that entry was written within 20 blocks of the current breakpoint — the lookback walks only those 20, and earlier writes are invisible.

Either one failing is a cold write. The request layers as tools → system → messages; a change hits one layer and voids that layer plus everything after it, leaving everything before it intact — so the earlier the change, the more it kills:

Change lands in	Voided (survivors in parens)
Tool definitions (any byte / add / remove / reorder)	tools + system + messages (all)
System prompt (any byte)	system + messages (tools survive)
`tool_choice` / thinking toggle / add-remove image	messages only (tools + system survive)

One hard floor: when the prefix at a breakpoint is below the model minimum, the write is silently skipped (Opus / Haiku 4.5 need 4,096 tokens, Sonnet 4.6 needs 2,048, older Sonnet / Opus 1,024).

Where the Breakpoints Go

Where breakpoints go needs no playbook — two rules fix it, both straight from the two hit conditions above:

Only at positions that stay byte-identical next turn. A breakpoint pays off only if its prefix hits next turn; if any byte in that prefix changes every turn, the breakpoint is a permanent cold write — wasted. This decides which positions are eligible.
Adjacent breakpoints ≤ 20 blocks apart. The lookback only walks back 20, so arrange the breakpoints as a ladder from tail to head, each rung ≤ 20 blocks from the next, and the walk-back always lands on a cached rung. One at the head alone won’t do — the moment the tail is > 20 blocks past it, you cold-write the whole head every turn. This decides why you need several.

Scan the request by these two rules and the byte-stable positions, front to back, are exactly these — the whole set of slots:

Position	What it is	Why it’s stable	Covers
① System prompt (+ tools)	the head — a breakpoint on the last system block	unchanged for the whole session	tools + system (30–80% of every request)
② Compaction point	the Task Context summary block	unchanged within a compaction epoch	the whole prefix up to the summary
③ Tail block	the last block before reminders	unchanged within this turn (new steps append after it)	the whole prefix up to this turn

Each rung has a job: ① is the biggest, most stable win; ② relays the lookback from tail back to head (without it, position ① eventually falls out of reach); ③ is the “write now, hit next turn” lookahead. (Between ② and ③ the code also adds a completed-round boundary rung when needed — an extra step when a single step exceeds 20 blocks; this exactly fills Anthropic’s 4-breakpoint cap, and with no summary it degrades cleanly to ①③.)

The reminder is not a slot. It changes every step — breaking rule 1 — so it’s deliberately appended after ③ and kept out of the cached prefix. Hence a hard ordering: place ③’s breakpoint first, then append the reminder — reverse it and the changing reminder joins the prefix and the next turn necessarily misses. (Placement is by role: system goes message-level, everything else block-level, to survive the adapter’s same-role merge.)

Keeping the Prefix Byte-Stable

With the four positions placed, the only remaining thing that breaks caching is volatile bytes in the prefix — two blocks with identical meaning but different bytes are two different blocks to the cache. The rules:

Freeze the head: no timestamps, current date, user / session IDs, or random nonces inside system or tool descriptions; push dynamic content later into messages.
Deterministic serialization: fixed JSON key order (Map / Set iteration order, Intl localization, and unsorted json.dumps are the silent killers).
Compute the tool list once per session: any reorder (even alphabetical vs definition order) voids all four breakpoints at once.

Conversely, some workloads cost more with caching — just don’t cache: one-shot / unique-per-request prompts, a head laced with high-variance content (fix the variance first), sub-minimum prefixes, and fan-out batches with no reuse.

Compaction’s Cache Consequences

Compaction is the only operation that rewrites the block stream itself (memory, JIT loading, sub-agent isolation all merely append, leaving existing bytes intact). So how the compactor edits directly decides whether the cache survives:

Head is immutable: compaction must never touch a block before position 1. Rewrite the system head and all four breakpoints invalidate on the next turn, with no partial recovery.
Replace whole blocks, never edit in place: a compacted round becomes one summary block (which then is position 2’s anchor); truncate a tool_result to a fixed length as a whole block, never mutate fields inside it — a sub-block edit changes every later prefix hash.
Cadence sets amortization: each compaction is a new cache epoch (one expensive write, many cheap reads). Compact at task boundaries, not every few turns, to keep the read:write ratio above ~10.

The gap is binary: on the same 30-step task, an append-only / whole-block compactor with position 2 in place holds hit rate ≈ 0.9, while one that rewrites the system head or edits tool_result bytes in place drops to ≈ 0.2 — roughly a 5× total-cost difference.

Measurement

Hit ratio is the headline metric:

hit_ratio = cache_read_input_tokens
          / (cache_read_input_tokens + cache_creation_input_tokens + input_tokens)

A well-behaved long task should hit > 0.85 after turn 3 and > 0.90 on sustained sessions; below 0.5 is a bug, not a tradeoff. The fastest self-check is the cold-start assertion: turn 1 has cache_creation_input_tokens > 0 and cache_read_input_tokens == 0; turn 2 has cache_read_input_tokens > 0. Zero reads on turn 2 means the prefix changed between turns — walk the previous section for non-determinism.

markPrefixCacheBoundary emits cache.breakpoints_placed on the Anthropic path (fields messagesCount / stepIndex / placedAt / lastRole) as the dashboard’s primary source: a placedAt length stuck at 1 (tail only) means the mid anchors never get placed and the lookback can’t reach the head.

← Compaction — that stable compaction point is where position 2 comes from.
Context Management — the attention budget; hit ratio is its cost dimension.
Operations / Dashboards — where cache.breakpoints_placed lands.

Sources

Prompt caching — Anthropic, Claude API docs (block model, 20-block lookback, invalidation rules, pricing).
Effective Context Engineering for AI Agents — Anthropic, 2026 (cache-aware compaction guidance).