Stream Timeouts & Tool Heartbeat

At a glance

agent.stream({ timeout }) exposes three timeout tiers. Two of them are per-tool overridable; the third — chunkMs — is global and turns out to be the real ceiling on any long-running tool. This page traces exactly why, and what the only valid heartbeat is.

Three timeouts, one merged abort signal

buildStreamTimeouts (packages/backend/src/agent/agent-loop.ts:442) assembles the object passed to agent.stream({ timeout }). The SDK reads three tiers plus totalMs:

chunkMs reset does not reset stepMs. They are independent timers: resetChunkTimeout() (stream-text.ts:1750) touches only the chunk abort controller; stepTimeoutId is armed once at the top of streamStep and only ever cleared, never re-armed. A stream that keeps emitting chunks resets chunkMs forever but still dies at stepMs counted from step start. chunkMs = “max silence between chunks”; stepMs = “hard wall on the whole step from its start”.

The decisive detail: all four abort sources are folded into one signal (stream-text.ts:732 — mergeAbortSignals(abortSignal, totalTimeoutMs, stepAbortController?.signal, chunkAbortController?.signal)). Any one of them firing aborts the entire stream, and that merged signal is also handed to the running tool (execute-tool-call.ts:120). So a chunkMs timeout does not merely stop reading the stream — it actively aborts the tool that is executing.

Each timer aborts with a TimeoutError DOMException labeled by tier (util/set-abort-timeout.ts): a chunkMs breach surfaces as Chunk timeout of 120000ms exceeded.

The exemption lifts toolMs, and misses the one that binds

tools: {
  [`${TOOL_NAME_EXECUTE}Ms`]: stepMs, // 300s
  [`${TOOL_NAME_TASK}Ms`]:    stepMs, // 300s
}

tools[{name}Ms] can only override the per-tool toolMs (request-options.ts — getToolTimeoutMs returns timeout.tools?.[name+'Ms'] ?? timeout.toolMs). This correctly raises shell’s and the subagent’s tool-level ceiling from 120s to 300s, so they fall back to the step-level bound instead of a tighter tool-level one.

But chunkMs is a single global value with no per-tool form — the exemption cannot touch it. So the tightest of the three constraints stays at 120s for execute and task too. The exemption’s intent — long tools should not be clipped early — is quietly defeated by a timer it never addressed.

chunkMs keeps ticking through tool execution

resetChunkTimeout() (stream-text.ts:1750) is called at the top of the transform that processes the merged stream — model chunks and tool-result chunks alike (stream-text.ts:2051). During a tool’s execution, the question is simply whether any chunk crosses that transform.

The sequence below shows it does not, for a tool whose execute is a plain async function:

Walking it: the model emits model-call-end; executeToolsFromStream forwards it downstream immediately (execute-tools-from-stream.ts:88), which reaches the outer transform and arms the chunk timer at the very start of tool execution. Then, on the same model-call-end, it awaits all tool executions (execute-tools-from-stream.ts:199). While that await is pending, nothing is enqueued unless the tool streams a result. clearChunkTimeout() only runs in the step’s flush (stream-text.ts:2274), after the tools have finished. So the 120s counts down through the entire silent execution.

Only a streamText-pipeline chunk resets chunkMs

There are two independent stream planes. chunkMs watches exactly one of them.

The only thing a tool can push onto plane A is a value it yields. There is no API to emit a bare data chunk into streamText from inside a tool — writeTransient is a side channel Zapvol opened on plane B, and plane B is invisible to chunkMs.

The current gap — shell streams stdout and still times out

shell.tool.ts:88 defines execute as a plain async function that returns a Promise. It does stream output — but via ctx.writeTransient(TOOL_STREAM, …) in its onStdout / onStderr callbacks (shell.tool.ts:100), which is plane B. No matter how many lines a build prints, the streamText chunk timer is never fed.

The effective ceiling on one shell command is therefore:

min(chunkMs 120s, per-tool 300s, stepMs 300s, shell self-managed hardWall) = 120s

Concretely: an operator can set timeout: 240, and the tool is deliberately exempted to 300s at the tool tier, yet any command whose wall-clock exceeds 120s is aborted at 120s regardless of output volume. The task tool has the identical shape — a subagent step that runs minutes without bubbling anything to the parent stream trips the parent’s 120s chunkMs.

The fix — an empty-payload heartbeat, keep the append ticker

A tool’s heartbeat is unavoidably a preliminary result — that is the only signal plane A carries. So make it carry nothing: yield a minimal, valid snapshot whose sole job is to re-arm chunkMs.

// execute becomes an async generator
execute: async function* ({ command, timeout }, { abortSignal, toolCallId, context }) {
  // display stays on plane B — fine-grained, append-style, transient (not persisted)
  // onStdout: ctx.writeTransient(TOOL_STREAM, { toolCallId, chunk: { … } })

  // heartbeat on plane A — an empty ShellOutput snapshot, < chunkMs apart
  yield { exitCode: -1, result: "", executionTime: elapsed }; // preliminary — re-arms chunkMs (exitCode is non-nullable, so -1 = "not exited")
  // …repeat while the command runs…

  yield { ...final, result: capShellOutput(final.result) };     // LAST yield = the real result
}

The contract that makes this safe is in execute-tool.ts (provider-utils): when execute returns an async iterable, every yielded value is emitted as { type: 'preliminary' }, and after the generator returns, the last value is re-emitted as { type: 'final' }. Downstream (execute-tool-call.ts:153), only the final output enters the model context; every preliminary is used purely for the heartbeat and the UI preview, then discarded. Two consequences:

• The last yield is the tool result. Heartbeats must be intermediate yields; the generator must yield the complete ShellOutput last and only then return. Yielding a partial snapshot as the final value would hand the model a truncated result.
• The heartbeat payload never reaches the model. An empty snapshot is safe.

Why empty and not the accumulated stdout: keeping the rich stdout on the writeTransient append (plane B) and sending nothing on the heartbeat avoids shipping the output twice. See the redundancy note below.

preliminary is not a data-part — they cannot be handled the same

A preliminary result becomes a tool-output-available chunk keyed by toolCallId (to-ui-message-chunk.ts:274), so on the client readUIMessageStream merges it into that tool call’s part, updating its output and marking preliminary: true. A writeTransient event becomes a free-floating data-{event} part handled by the data channel. The two land in different places in the message tree and are handled by different renderers.

This is why a naive dual-write — keep writeTransient for stdout and put stdout in the heartbeat — is redundant: the client receives two copies of the same output through two structurally different channels, with no way to process them uniformly. The empty-payload heartbeat sidesteps it: the preliminary carries no content, so it competes with nothing. The client simply ignores preliminary: true outputs for the shell tool (a one-line guard; an empty output renders nothing anyway).

The alternative, and why not

The SDK-native path is to drop writeTransient entirely and stream a full accumulated snapshot on every stdout tick — one channel, no redundancy, preliminary doubling as both display and heartbeat. The cost is real: overwrite (snapshot) semantics instead of append, and O(n²) bandwidth for chatty output (each tick re-sends everything so far) versus O(n) for the append ticker. For build and test output that argument favors keeping the append ticker and using a minimal heartbeat, which is also the smaller change and reuses the existing TOOL_STREAM renderer.

Compaction runs in prepareStep — a different timer story

Context compaction runs inside prepareStep (agent-loop.ts → stepCompactor.apply), before the step’s model call. Its interaction with the two timers is the opposite of a tool’s, because of where each timer is armed in streamStep (stream-text.ts):

1. setAbortTimeout({ label: "Step" }) — arms stepMs at the top of streamStep.
2. await prepareStep(...) — compaction runs here.
3. first resetChunkTimeout() — only fires inside the transform that processes model chunks, i.e. after the model call starts producing.

Two consequences:

• chunkMs cannot fire during compaction. The chunk timer isn’t armed until the first model chunk crosses the transform, so a slow compaction never trips chunkMs. (This is the mirror image of the tool case, where model-call-end has already armed the chunk timer before the tool runs.) The SSE connection during that silence is held open by the transport-level withSseKeepalive comment ticker (apps/server/src/lib/sse-keepalive.ts), not by chunkMs.
• stepMs is charged for compaction time. Because the step timer is armed before prepareStep, compaction’s wall-clock counts against stepMs and cannot be reset. Left unaddressed, one near-limit compaction would eat into the budget the execute / task exemption is supposed to hand the tool. So the effective stepMs is work budget (300s) + compaction allowance (60s) = 360s, where the allowance equals compaction’s own soft-timeout (below), keeping the tool’s work budget intact.

Never abort compaction — bound it with a soft-timeout instead

Compaction is a prerequisite, not optional work: if a timeout aborted it mid-flight, the context would still be over-length and the step would fail anyway (ContextLengthError, or a request rejected for exceeding the window). Aborting just moves the failure. And prepareStep has no abortSignal parameter, so stepMs firing during a hung compaction cannot even interrupt it — execution is parked on await prepareStep, so a stalled summarizer LLM call would hang the whole turn unbounded.

The fix is a soft-timeout that degrades, never a hard abort. The FLOOR summarize LLM call (summarizer.ts) is given its own AbortSignal.timeout(COMPACTION_SUMMARIZE_TIMEOUT_MS) (60s). On timeout it falls back to the summarizer’s existing deterministic static checkpoint (most-recent user request + action count) — which still bounds the context, so the step remains feasible. Compaction therefore always completes and always shrinks the context; it just downgrades to a thinner summary under a stalled model. Offloaded tool outputs remain recoverable via view_tool_call.

While that (possibly slow) compaction runs, prepareStep emits a transient AGENT_STATE: "compacting" progress event (gated on predictedTokens >= trigger, the same threshold the engine uses) so the operator sees “compacting context”, not a frozen turn. The next step’s start-step → executing state naturally supersedes it.

Where each fact lives