Context Compaction

The message stream is never rewritten

A 20-step run with file reads, grep, and code execution routinely reaches 100K+ tokens. That overflows the model’s context window — and even below the limit, a long context is not uniformly usable: attention degrades as the window fills, the effect known as context rot, so effective context is smaller than nominal. Either way the model cannot be given the whole history; it is given a reduced view, and producing that view — folding, reducing, and reordering the history under a token budget — is what compaction does. Compaction edits what the model sees, deliberately, on every call.

What it must not edit is the record that view is derived from. Two views coexist over one task:

the record (task_message) — append-only, never rewritten; the full history the user sees, and the source every projection is built from;
the model’s input — a projection over that record, reduced as the budget requires and discarded after the call, never written back.

Compaction operates only on the second. Keeping the first immutable is what makes everything downstream work: the user always has the true history; anything reduced out of the model’s view still exists in the record it came from; and the projection is deterministic, so a resume or retry re-derives a byte-identical view rather than drifting.

Two physical facts form the foundation; everything else is derived:

task_message — immutable, append-only raw history. Any compacted content is recoverable by re-reading the original. It is the keyframe source every compaction is re-derived from.
The snapshot store — append-only, content-addressed. It holds the non-reproducible LLM outputs (per-tool compactions, floor summaries) keyed by content address, plus one cross-turn calibration anchor (meta:anchor, the prior step’s real total token count that seeds the next step’s size prediction) — the store’s one fixed-key, non-content-addressed metadata entry.

Every compaction decision — what to restore, shrink, or drop, and where the boundary sits — is recomputed each call from the raw, the store, and the budget; the decision itself is never persisted. What the store does persist is the content-addressed reduced forms and summaries — the inputs to that re-derivation, not the decision. This eliminates stale-decision bugs and makes interruption self-healing: a crash or HITL pause loses only in-process scratch, and the next call re-derives the same view.

Two levers on one axis

Compaction and prompt cache are often framed as opposed. They are not. Pure raw has the highest hit rate — it is append-only, the prefix never changes, and every call hits. Its problem is size: each call re-reads an ever-growing history at the cache-read price and eventually overflows the window. Compaction therefore serves size, not cache: it holds the prompt below the window and accepts a small cache cost to do so.

Lever	What it does	Constraint
Compaction	Shrinks prompt size; fired on pressure	Bounded by information loss
Prompt cache	Re-reads a sent prefix at read price	5-minute TTL; 1.25× write premium on a miss

Sonnet 4.6 pricing (2026): input $3.00/M, cache-write $3.75/M, cache-read $0.30/M. Break-even 3N = 3.75 + 0.3(N-1) solves to N ≈ 1.28 — a prefix must be read at least twice to net positive. This break-even is why compaction is threshold-gated rather than eager: it fires only when size threatens the window, leaving the prefix stable across the steps of a turn so the cache continues to pay off. See Cache vs Compression for that axis in the abstract.

One operation, run at every model call

Compaction is a single operation, run inline at every model-call boundary. Its three moves are always the same:

Reconstruct the message stream into addressable parts (one tool call, text block, or reasoning block each).
Derive the compacted view — restore, measure, and, only under pressure, continue reducing.
Project that view back into the messages sent to the model.

It is a threshold-gated checkpoint model: compaction fires on token pressure, not turn count. The unit of compaction is a part, not a turn; that is the source of the properties below. The only turn-end work is persisting the cross-turn calibration anchor once the turn’s messages are durable.

Restore, then measure, then continue

The order is load-bearing: restoring must precede the budget check, because raw parts that ignore prior compaction do not reflect the true size.

Restore (unconditional, free). Reconstitute the prior state — fold the already-covered prefix into its one summary part, then swap in every stored per-part compacted form. Only after this is the measured size accurate.
Measure against the threshold. If the context is at or under trigger, return the restored view unchanged; restoring alone sufficed (or an anti-thrash gate is holding).
Continue only if still over. Apply the reduction ladder oldest-first, storing each new reduced form.

Three properties follow:

Restore is always safe. The conversation grows monotonically, so any form compacted in a prior pass is still wanted. Restoring every stored form on every call keeps compaction monotonic and therefore deterministic: identical raw plus identical store yields a byte-identical prefix, preserving cache hits.
Reduce is the only budget-driven step. Restore and the threshold check are pure projection; only reduce makes new reduction decisions, and only when the measured size exceeds trigger.
The threshold does not re-sum the array. It reads a running prediction — a real-token anchor plus the new increment (see Real-token calibration) — and reduce takes that same value as its starting point, subtracting each part’s saving as it proceeds. The hot path never walks the whole array to total it.

Two boundaries, one logic

The message source is the only thing that varies between the two boundaries; the logic does not.

Boundary	Source	Role
preflight (turn start, off the hot path)	the full raw history	Restore a bounded checkpoint: folding on the pristine raw collapses the earlier history into the summary, so the context entering the loop is `summary + recent window`, bounded regardless of history length. It does not reduce.
step (every model step, including the first)	the bounded checkpoint plus this turn’s new growth	Run the full operation — restore, measure, reduce, including the floor. Turn-start overflow is handled incrementally by the first step.

Preflight folds on the pristine raw, not on the step’s already-compacted view, because the fold’s coverage boundary and the full tool identifiers are present there — so the earlier history collapses into the summary. This is what keeps the checkpoint bounded on a long horizon: were the full raw history handed to every step instead, the runtime’s per-step reassembly would re-parse and re-fold the entire history on every step, growing without bound. At the step boundary the fold sees a context whose old span is already summarized, recognizes the materialized ## Task Context head as a no-op, and reduces only this turn’s new growth.

The hot path carries no synchronous LLM except the floor. Tool-compaction’s LLM cost is paid off it: each large tool output’s compacted form is computed eagerly in the background as the tool finishes, so at both boundaries restoring a tool is a cheap read (hit → use it, miss → skip). The one remaining synchronous LLM is the summarize floor (folding the oldest prefix), which cannot be precomputed. The floor is available on every step: an over-budget step folds in place instead of throwing and re-running the turn, so a hard-limit error now signals only that even the floor cannot fit — the true ceiling.

The reduce ladder: near-window protection, then breakthrough

Reduction runs only when the measured size exceeds trigger. It then walks a four-tier ladder, escalating a tier only if the previous one left the context over trigger. The organizing principle is a hard-protected recent window: ordinary compaction never touches the last N steps, and only a last-resort tier breaches them.

The near window is defined by step, not tokens — COMPACTION_PROTECT_RECENT_STEPS (default 5), where a step is one model generation. Regardless of a step’s output size, the last N steps are never compacted incidentally, so content the agent has just read (a skill body, a file) is not evicted on the following step and forced to be re-read.

Tier	Action	Near window
①	Shrink old tools outside the near window, oldest-first, toward `stop` (a generation is finished as a unit, never bisected)	protected
②	Floor summarize the old prefix outside the near window (the fold boundary is held outside the near window)	protected
③	Breakthrough shrink — ① and ② are exhausted and the context is still over `trigger`: shrink the whole array, near window included	breached
④	Breakthrough floor — ③ still cannot fit (the near-window tools have no compacted form or are already shrunk): summarize with no near-window protection, the fold as last-resort floor	breached

Tiers ① and ② respect the same near-window boundary; only ③ and ④ breach it. The raw tail that ② keeps is therefore max(the token window, the step window of the last N steps), whichever is larger. The floor summarizes at most once per compaction — progressive folding spreads the LLM cost across checkpoints — so ④ fires only when ② did not fold that pass.

Two consequences follow, and they explain why compaction sometimes settles above stop:

trigger is the gate; stop is only ①’s target. The entry to reduction and every escalation are gated on > trigger. The shrink sweep aims for stop while it runs, but once the context drops under trigger the ladder accepts it and stops. The band (stop, trigger] is a hysteresis dead zone: accepted deliberately rather than paying for an expensive, lossy summarize to reclaim the last stretch below the danger line. The floor, too, fires on trigger, not stop — a shrink that fell short of stop does not by itself trigger a summarize; only remaining over trigger does.
The near window is the achievable floor. Tiers ① and ② cannot reduce below what the summary plus the last N steps occupy. If stop sits below that floor — common on a small window, where the protected steps alone approach it — the context cannot reach stop by ordinary means; it settles in the dead zone, and only a breakthrough tier can go lower. This is the intended behavior, not a failure: the recency guarantee is worth more than hitting stop exactly.

Graded levers — the rung × scope grid

The ladder chooses, per part, the cheapest lever that meets budget.

Axis A — rung (by in-context density × compute cost):

Rung	Lever	In-context loss	Cost	Recovery
R0	keep raw	none	0	—
R1	restore a stored compacted form	small (gist kept)	0 (already stored)	`view_tool_call` (offloaded)
R2	drop — a true purge (only when a tool declares its output disposable, e.g. `reflect`)	in-context content reduced to an empty shell	0	not recoverable by the model (raw stays for admin / replay only)
R3	tool LLM compact — computed eagerly in the background, read at reduce time	small (gist kept)	0 at reduce time (LLM paid in background)	`view_tool_call` (offloaded)

R1 and R3 converge: both read a stored tool-compacted form. The LLM cost is paid eagerly in the background when a tool finishes; at reduce time the form is only read back. The reduce-time choice is therefore R3-read (hit, keeping the gist) versus skip (miss, leaving the part for the floor). Two points are worth stating precisely:

A shrink miss is a skip, not an automatic drop. If the background form is not ready, the output is below the precompact threshold, or the tool has no compaction rule, restoring returns nothing and the part is left untouched; the floor summary is the only mechanism that will eventually absorb it. The compaction layer never drops a part on its own initiative.
drop is a true purge, and only a tool requests it. It is not a recoverable index card: the part becomes a minimal legal shell — the call identity kept so the tool-call/result pairing stays intact, the input and output content elided — with no offload and no recovery pointer. The raw remains for admin and replay, but the model cannot recall it. It is reserved for outputs a tool explicitly marks used-once-and-discard. R2 and R3 apply only to tools at or above the precompact threshold; smaller tools ride into the floor summary with everything else.

Axis B — scope (by coupling): tools are self-contained (reduced individually); text and reasoning couple within a turn (a signed reasoning ↔ tool-call binding), so a whole turn-span is summarized or dropped together, never split.

The R3 tool lever is defined per tool: shape-preserving (the output schema is kept, only content fields replaced), gist-keeping (line numbers, outline, key identifiers), and recoverable (raw I/O offloaded, the compacted form carrying the pointer). Both shipped tool families reduce deterministically, without an LLM: grep / glob / ls take a top-N slice, and read / write / edit keep head + tail lines with the middle elided. The framework also supports an LLM-summarized tool form (kind: "llm"), unused by the current tools — so the only synchronous LLM in compaction is the floor summary.

Addresses, not ids

Nothing is addressed by a message id — the message stream carries no stable one in-loop, and re-deriving from raw would re-mint any that did exist. Content is addressed by what it is:

a tool part by its toolCallId (present in message content, and preserved across the round-trip to model form);
a text / reasoning part by a canonical hash of its content.

This is what makes one operation over two message sources sound: the same logical content yields the same address whether reconstructed from the stored form or from the in-loop form. A form minted at preflight is found again at the next step with no id translation, because there are no message ids in the address space at all.

Decisions are re-derived every call

The only physical truths are the immutable raw history and the append-only store. Everything else — what to restore, shrink, or drop, and where the boundary sits — is recomputed each call from the raw, the store, and the budget. What the store holds is the reduction outputs — per-part compacted forms, floor summaries, and the calibration anchor — not the decisions themselves; so decisions always track the current budget, and re-derivation over a monotonic store is idempotent — the property the failure modes below rely on. Editing an earlier message is safe for the same reason: because addressing is by content and the view is re-derived from raw, an edit simply changes that content’s address and the next call folds the new raw; there is no persisted anchor to dangle.

Real-token calibration

The gate must judge accurately whether the context exceeds trigger. Estimating the whole message array with a chars/4 heuristic accumulates error as the conversation grows, so the engine instead anchors on the provider’s real prompt tokens and estimates only the new increment.

predictedTokens = prevRealTokens + increment       // used by every gate / stop / exhaustion check

prevRealTokens — the last step’s measured whole prompt plus output, taken directly from provider usage with no estimate; real overhead and output are included, not subtracted.
increment — the only estimated quantity, the slice whose real value is unknown before the call: within a turn, this step’s tool-output characters; at turn start, the new user input.

Because the bulk (old, unchanged parts) is anchored on a real measurement, only the fresh increment is estimated. The anchor already includes real overhead, so the budget deliberately does not subtract an estimated overhead again; double-counting would reintroduce the error.

Within-turn: each step’s real usage refreshes the anchor, so the next step anchors on real usage and estimates only the new tool output.
Cross-turn: the anchor is persisted at the terminal step (meta:anchor) and seeded back at the next turn’s preflight, so the first decision anchors on the prior turn’s last real usage. The anchor lives in real token space and is portable across turns even if the model or overhead changes. A cold start (no usage yet) leaves the anchor at 0, degrading to pure estimation.

Anti-thrash

Two ineffective compactions in a row (each saving below a threshold, or making no progress) stop further attempts for the turn, so the engine never loops compact → save nothing → compact. Only an effective compaction resets the counter. When the gate holds off a step that is still over trigger, the operation returns without escalating: trigger sits below the real window (the hysteresis margin), so an over-trigger-under-window step proceeds normally, and a genuinely over-window step is caught reactively by the provider’s own reject, which re-enters error-recovery. Throwing proactively on a futile gate would only re-enter the same futile loop.

Budget: trigger and stop

trigger and stop are computed on window-caliber space — the window minus only the output reservation, not the prompt overhead:

wEff    = contextWindow − OUTPUT_RESERVED_TOKENS
trigger = wEff × COMPACTION_TRIGGER_RATIO      // exceed → compaction fires   (default 0.65)
stop    = wEff × COMPACTION_STOP_RATIO         // ①'s shrink sweep aims here  (default 0.50)

Overhead — the system prompt and tool definitions — is deliberately left in. The gate compares against a real-token anchor that already includes measured overhead (see Real-token calibration), so subtracting an estimated overhead from the window as well would double-count it. Both sides carry overhead; the comparison stays consistent. (instructionsTokens and toolsTokens are still measured — for admin display and the legacy message-space budget — but the gate does not subtract them.)

The two thresholds answer two different questions.

trigger is a quality ceiling, not merely an overflow guard. Effective context is smaller than nominal context — attention degrades well before the window is full (context rot) — so trigger targets the fraction where quality begins to suffer, which sits below the nominal limit and below the provider’s hard reject. Setting it conservatively trades nominal capacity the model cannot use well for output quality it can. It is a per-window ratio because the effective fraction, not the absolute size, is what matters.

stop sets the hysteresis headroom. The gap trigger − stop is how much one compaction reclaims — the runway of steps before the context climbs back to trigger. Too small a gap re-triggers almost every step (churn, and a prompt-cache rewrite each time); too large a gap makes each compaction aggressive and discards context that was still affordable. Because both are ratios of wEff, the dead zone scales with the window and never clamps to zero; the trade-off is that a fixed ratio yields different step-headroom on different window sizes, so a very small window runs a tight dead zone.

This distinction matters when reading telemetry. The gate compares on window-caliber tokens: a step’s recorded input tokens already include prompt overhead, and trigger is measured on the same basis (overhead left in), so the two compare directly — no overhead adjustment. (Only the legacy message-space budget subtracted overhead first; the compaction gate does not.)

The floor summary constitution

The floor (tiers ② and ④) folds the oldest prefix into one dense summary part, rendered as a ## Task Context block at the head of the prefix and re-sent on every step. Because it is re-read on every future turn, it is written as a digest, not a transcript — the process (how a thing was done, the tool steps, the search trace) is dropped by design; a load-bearing conclusion, user intent, deliverable, or safety rule is not.

One admission gate: keep only what cannot be cheaply re-fetched. Current file contents are re-readable, so a bare “files I read” map earns nothing — only a re-readable file path is a real handle; a URL is process (there is no fetch tool, so a link recovers nothing). What has no real-time source — the original intent, a decision and its rationale, an approach ruled out, a constraint the user imposed — must be preserved.

Structure — append-only fact sections, then one overwrite. Every fact section re-emits each prior line verbatim and appends this turn’s delta; only ## Current state is overwritten. Each fact is one topic: statement line, where topic is a stable kebab key — reusing it supersedes that fact. The most damaging failure is reusing a key for a different subject, silently overwriting a live fact, so the summarizer runs a topic self-check before writing.

Section	Role
`## Requests`	Every distinct user ask, ordinal `#1` / `#2` / … assigned once and never renumbered, load-bearing wording verbatim — the intent log.
`## Deliverables`	A file artifact shipped via `complete`: path + a one-line note (keep the path, never the contents). Never dropped.
`## Decisions`	A choice made (approach / tool / format / convention) + why.
`## Constraints`	A standing limit or quota, or any safety / privacy / security rule in force. Safety rules are kept verbatim and never dropped.
`## Ruled out`	An approach tried and abandoned + why + the error verbatim — the anti-re-explore spine.
`## Findings`	A discovered truth that cannot be cheaply re-read. A time-bound value (a status, a “latest” count) carries an `(as of <date>)` anchor so a later read re-verifies it. Detail scales with role: incidental → one terse line; the deliverable answer itself → kept in full.
`## Archived`	Where an aged Finding (or, under sustained pressure, an aged non-safety Decision) demotes — a `see <file>` pointer if a file backs it, else a terse gist. Demotion, not deletion.
`## Current state`	Overwritten each fold: Goal · Deliverable (delivered vs still owed) · Status (blockers / awaiting input) · Next.

Confidence is tagged only by exception — verified is the unmarked default; (reported) when a source merely stated it, (assumed) when inferred.

Priority floor. The head is re-sent every step, so a token cap (ROLLING_SUMMARY_BUDGET_TOKENS, 2500) keeps it bounded on a long task. Once the carried summary exceeds it, the summarizer is told to demote, never delete, lowest tiers first:

P0 — safety / privacy constraints + the request log: never demote, kept verbatim.
P1 — deliverable file paths: the note may shrink, but the path stays (deleting it strands the asset).
P2 — non-safety decisions / constraints / ruled-out: supersede in place, then the oldest no longer bearing on the goal → an ## Archived pointer.
P3 — findings: oldest → an ## Archived pointer.

“Keep” means keep a recoverable handle — a path, a pointer, a one-line headline — never the full artifact; detail lives in the files the agent re-reads.

Two defenses against decay. The fold re-compresses the prior summary against the new prefix, so early information could erode across many folds. Against that: the sticky no-delete precedence above, and the raw backstop — the original turn is always in task_message, and every turn’s preflight re-folds from that pristine raw rather than from a summary of a summary, so a span is re-derived from source each turn instead of compounding drift. If the summarizer model is unavailable or times out, the floor degrades to a deterministic static checkpoint (the most recent user ask + an action count) rather than throwing — it is the last guard before overflow, so it must always produce something.

How this maps onto Claude Code’s compaction

The two layers here — the in-loop tool layer and the semantic summary layer — are the two mechanisms Claude Code’s compaction uses. The shape is shared; the differences are parameters, not architecture.

Tool layer — a refinement of microcompact. Claude Code’s microcompact clears old tool results to a placeholder when context grows: blunt, lazy, fired on the threshold. This engine shares the concept (in-loop, tool-targeted, size-triggered, oldest-first, recent steps kept raw) and adds two things. First, tool-aware compaction: each tool keeps its own salient content — a grep’s matches, a read’s key lines — a targeted reduction, with the blunt clear demoted to a last-resort drop. Second, eager and durable minting: the compacted form is computed off the critical path and stored, so it survives a crash or HITL pause and re-applies deterministically. The cost: this precompute runs for every large tool output even when the budget is not hit that turn — a cost Claude Code’s lazy clear avoids.

Semantic layer — the same incremental fold at a different cadence. Both are Σn = fold(Σn-1, new). Claude Code’s auto-compact does not re-read the original raw; after its first compaction the raw is already gone from context, so its second compaction folds summary + new work, exactly as this engine folds summary + new prefix. One capability is unique to this design: because the raw history is an append-only source every checkpoint re-derives from, this engine can re-summarize a span from the original raw — a true drift reset — which auto-compact structurally cannot.

Compaction is not loss

Every reduced form is recoverable from the raw, with one deliberate exception:

Source of truth. The raw history is never deleted.
view_tool_call re-reads a single compacted tool call’s full input and output from its offloaded raw I/O. This is the recovery channel for shrink (R1 and R3) compaction.
drop is the exception — a true purge. A dropped part carries no recovery pointer, and the model cannot recall it. This is intentional and occurs only when a tool declares its output disposable; the raw remains for admin and replay, but not for the model.

Before restoring, a free pass replaces re-sent historical images and inline non-path files with a text placeholder; path-file references are kept, being small and re-readable on demand.

Failure, HITL, crash — self-heal

In-loop state is ephemeral. A HITL pause, abort, or crash mid-turn loses only in-process scratch.
Nothing is serialized across a streaming boundary. The tool store is durable and content-keyed, and the calibration anchor is a single persisted row; there is no in-memory decision state.
Resume re-derives. The next call rebuilds from the raw history plus the store — the same view from the same inputs. Alignment is re-derivation from a shared raw source, not state inheritance, which is what lets any interruption self-heal.

Data model

Three durable tables — one immutable raw source and two content-addressed projection stores; the raw is shared and never rewritten.

Table	Role	Shape
`task_message`	Source of truth / recovery source — pure raw, append-only	user / assistant rows
`task_compaction_snapshot`	Content-addressed store — one row per stored reduction, upsert by key	`(taskId, key)` → entry; the key is a part address, a summary coverage, or `meta:anchor`
`task_tool_compaction`	The `toolCallId`-keyed tool store — written eagerly on tool finish, read by the shrink lever	`(taskId, toolCallId)` → compacted form + offload pointer

The snapshot store is append/overwrite-by-key: a monotonic escalation (a part’s shrink → drop) overwrites its prior form, and re-derivation is idempotent. Because the store is re-read every turn, previously-minted summaries, tool-compacts, and drops survive across turns. Isolated paths (subagent / chat / benchmark / tests) run the store in-memory with no persistence: correct within a turn, re-derived from raw each new turn.

Terminology — turn vs step

Term	Meaning
turn	A `(user, assistant)` message pair persisted in the raw history
step	One model call plus its tool invocations, within one turn

Preflight runs once per turn (off the hot path); the compaction operation runs at every step, including the first. Near-window protection counts steps, so “the last N steps” is a tail of the reconstructed parts.

Cache vs Compression — the size ↔ cache axis in the abstract, and the industry survey of trigger / preservation / recovery strategies this implementation sits within.
Memory Design — what persists outside the window need not be compressed, only retrieved; complements the summary floor.
Tools — view_tool_call is a tool the agent calls to recover a compacted output.

Sources

Effective Context Engineering for AI Agents — Anthropic, 2026
Compaction — Anthropic, Claude API docs