Four Loci of Update

Learning signals indicate that an Agent should change, but not where that change should occur. The primary basis for choosing an update method is not how long an experience has persisted, but whether the target capability can be naturally represented by a particular medium. Facts and experience are suited to knowledge documents; strategies that can be clearly expressed in language belong in Prompts or Skills; precisely executable procedures and constraints should be encoded as programs; and high-dimensional capabilities such as perception, language style, and implicit strategies must enter model parameters. Figure 8-3 shows these four methods and their relationships.

Table 8-2 provides a concise comparison. The four methods are not mutually exclusive: a medical-imaging Agent relies on parameters to identify lesions, uses a knowledge base to provide current guidelines, and employs code to calculate risk indicators. A customer-service model derives its natural tone from post-training, obtains enterprise-specific policies from knowledge and Skills, and relies on server-side code to enforce critical compliance requirements.

Table 8-2 Applicable boundaries of four continual evolution methods

Update method	Suitable content	Primary advantages	Primary limitations
Experience knowledge base	Facts, experiential patterns, exceptions, and sources	Fast updates, traceability, on-demand retrieval	Depends on retrieval and correct model application
Prompt and Skill	Linguistically expressible judgment principles and operating procedures	Interpretable, controllable scope	Prone to bloat, conflict, or being ignored
Programs and Harness	Deterministic procedures, tools, and hard constraints	Testable, stable execution, low cost	Higher development and maintenance costs
Model parameters	High-dimensional perception, generation style, and implicit strategies	Strong generalization, low inference overhead	High update and regression costs

Consolidating Experience into Knowledge

The most lightweight form of evolution is to organize recurring experience from multiple runs into retrievable knowledge documents. The “experience knowledge base” described here shares storage, indexing, and retrieval technologies with Chapter 3, but differs in its knowledge sources and verification objectives. Chapter 3 primarily extracts “what the user and the world are like” from user conversations, documents, and datasets; this chapter extracts “what should be done under which conditions” from Agent action trajectories and outcomes. For example, “This airline requires special meals to be reserved twenty-four hours in advance” is domain knowledge, whereas “Check the special-meal deadline before booking to avoid discovering only after payment that the request cannot be fulfilled” is action experience.

Raw trajectories are unsuitable as formal knowledge units. They are lengthy and noisy, containing raw tool output, incidental detours, and environmental details. A more robust system retains three layers of data: immutable raw trajectories for auditing; per-run analyses recording the outcome and candidate lessons; and comparisons, clustering, and induction across multiple similar trajectories to produce future-oriented Markdown knowledge documents. A formal document typically specifies applicable scenarios, recommended strategies, prohibited practices, exceptions, evidence sources, and the latest verification time rather than retelling the complete course of a single task.

This design shares the same two-stage principle as User-as-Code in Chapter 3. User-as-Code first appends conversational facts to an immutable log and then periodically rebuilds a structured user model. Experience learning should likewise preserve evidence first and generate mutable knowledge offline afterward. Figure 8-4 illustrates this process. Separating recording from organization prevents a single accidental success or network failure from immediately changing the Agent, while allowing the system to identify common patterns only after observing multiple successes and failures.

Experience documents are not simple trajectory summaries. Transferable content emerges from comparison: what successful trajectories of the same type did, what failed trajectories lacked, in which environment versions a strategy was effective, and under which prerequisites it failed. Chapter 3 has already introduced knowledge extraction, clustering, and retrieval, so this chapter does not repeat those algorithms. Instead, it focuses on how trajectory evaluation becomes a condition for extraction and whether the extracted knowledge improves performance on subsequent tasks.

A complete knowledge-distillation pipeline can be divided into five steps. First, preserve immutable trajectories and environmental outcomes. Next, produce a structured analysis for each run, listing the task type, required capabilities, observed strategies, errors, and exceptions. Then aggregate runs by task family and build an evidence table showing which trajectories support or contradict each candidate pattern. Only candidates that meet the support threshold enter formal documents. Finally, evaluate transfer on new tasks that were not used during distillation. Keeping formal knowledge separate from candidate analyses allows the system to generalize again without altering the original evidence and to revoke a conclusion precisely when the environment changes.

GAIA experience learning provides an intuitive example. GAIA contains multistep problems that combine search, web reading, file processing, and computation, while AWorld provides the environment for running Agents, invoking those tools, and recording trajectories: the former is like the exam, and the latter is the exam room and laboratory record system. A simplistic approach generates a strategy summary and immediately vectorizes it after one successful run. A stricter implementation first uses a GAIA answer verifier or another environmental verifier to label runs as successful, partially successful, or failed, and then compares multiple paths within the same task family. Successful trajectories contribute candidate strategies, failures contribute exclusionary knowledge, and partial successes reveal which segment worked and which still failed. The natural-language reflection proposed by Reflexion can help generate candidate lessons, but reflection itself is not evidence. Only content consistent with environmental outcomes, supported across trajectories, and showing positive transfer on new tasks should enter formal experience documents.

Encoding Experience as Instructions

An experience knowledge base provides reference material for an Agent, whereas Prompts and Skills are more prescriptive. When multiple trajectories repeatedly reveal the same strategic error, and the pattern can be clearly expressed in natural language, the system can elevate it from “experience for reference” to “a rule that must be followed.” Rules that apply to nearly all tasks are suitable for inclusion in the system Prompt; complex procedures that apply only to a particular domain, project, or tool are better written as on-demand Skills or project instruction files.

Prompt learning serves a different role from the Prompt engineering discussed in Chapter 2. Chapter 2 explains how to write structurally clear, cache-friendly Prompts; this section addresses what production feedback is sufficient to trigger a Prompt revision and how new rules should be validated before deployment. Revision should not mean repeatedly rewriting the entire system Prompt. A more reliable approach is to generate a minimal diff from a group of similar failures, specify the rule’s scope, check for conflicts with existing rules, and evaluate it against both the boundary cases that triggered the failures and a retention set of old tasks.

In a 2025 long-form post, Andrej Karpathy provisionally called this possible new paradigm System Prompt Learning. His summary was that pretraining primarily learns knowledge and fine-tuning primarily shapes habitual behavior, while another kind of human learning occurs when we solve a problem and leave an explicit note to our future selves: “Next time I encounter this kind of problem, I should try this approach first.” He compared an LLM without such a notebook to the protagonist of the film Memento and noted that System Prompt Learning and reinforcement learning both improve behavior from experience but use different update algorithms—the former edits text, while the latter changes parameters through gradient descent. His example was an instruction in Claude’s then roughly 17,000-word system Prompt requiring the model to number and explicitly count words, letters, or characters before answering, precisely to handle questions such as “How many rs are in strawberry?”

In an Agent system, this means turning lessons that can be expressed in language into candidate rules that future runs can read directly. Compared with a scalar success/failure result, an evidence-backed diagnosis can identify whether the error was in identity verification, tool selection, or escalation boundaries, enabling a more targeted candidate change. Karpathy’s observation that a knowledge-guided review is a higher-dimensional feedback channel than a scalar reward helps explain the method’s potential data efficiency. Richer information is not automatically correct, however: one user’s feedback may apply only to that customer or an outdated policy, so clustering, scope analysis, and regression testing remain necessary.

Several established approaches automate Prompt optimization in different ways. DSPy treats a program composed of multiple language-model calls as an optimizable object and searches instructions and examples on a development set. OPRO asks a language model to propose new candidates from the history of Prompts and their scores. GEPA uses natural-language reflection over failed trajectories to generate and select complementary candidate Prompts. These methods primarily perform batch optimization on offline evaluation sets; minimal production diffs are closer to continual maintenance, triggered by newly observed boundary cases and designed for provenance, auditing, and rapid rollback. In practice, offline search can establish a strong initial version, followed by case-by-case patches for long-tail production rules.

For example, an airline customer-service Agent may escalate to a human too early when users challenge a policy. Trajectory evaluation shows that it violates no rules but lacks compliant flexibility. A candidate patch can require the Agent to explain the policy first, identify the user’s actual goal, and seek permitted alternatives, escalating only when the user explicitly requests it or the issue genuinely exceeds the Agent’s authority. If the new rule reduces unnecessary escalation but causes the Agent to continue handling safety incidents that should be escalated, it has failed regression testing. The value of system Prompt learning lies not in automatically appending more text, but in continually clarifying the scope of rules through production boundary cases.

Skill learning follows the same principle, but with a more localized scope. A Skill can be understood as an on-demand operating manual for a particular job: if multiple experiences collectively form a complete insurance claims process, the system can generate or revise the corresponding Skill. A candidate Skill should not merely summarize one conversation; at minimum, it should specify when to load, prerequisites, operating steps, known pitfalls, validation methods, and source trajectories. The system first searches the existing Skill library for similar capabilities, preferring a local patch when the same process already exists and creating a new directory only for a genuinely independent capability. This prevents the library from filling with manuals that differ in name but duplicate one another. Anthropic’s Skill Creator demonstrates a draft–test–evaluate–revise loop. It addresses how to create and improve a Skill; the harder questions remain what operational evidence is sufficient to trigger creation, how to resolve conflicts, and whether the revision passes domain-specific and old-task regression tests.

Encoding Experience as Programs

When experience describes operations that are stable, repetitive, and verifiable, it is inefficient to have the model reread documentation and reason through them each time. A more appropriate approach is to compile the experience into workflows, tools, or Harness code, turning a one-time exploration into a repeatedly executable program. Chapter 5 explained how Coding Agents read and write files, run tests, and generate systems; this section focuses not on general code generation, but on how an Agent modifies future versions of itself based on its own trajectories.

The modifiable objects extend far beyond new tools. At the operation layer, browser trajectories can be compiled into parameterized workflows, or adapters can be generated for changing APIs. At the control layer, tool routing, retries, circuit breakers, and context compression strategies can be modified. At the validation layer, parameter checks, state validators, and regression tests can be added in response to production failures. At the architecture layer, a Reviewer Agent can be added or the information flow between planning and execution can be changed.

Browser workflows illustrate the value of programmatic experience. They are analogous to recording a spreadsheet macro. The first time an email is sent, a multimodal Agent uses an observe–reason–act loop to find the compose, recipient, subject, body, and send controls. For another email, the process is unchanged; only the recipient and content differ, so there is no need to call the model again to rediscover the entire path from pixels and the DOM. The system compiles the first exploratory trajectory into a small program containing parameters, state checks, and version information.

In the browser setting, the knowledge-distillation process shown in Figure 8-4 becomes a more concrete lifecycle:

Capture the trajectory: Record navigation, clicks, text entry, and drop-down selection, together with action parameters, the current URL, and element-locator evidence such as XPath, CSS, id, role, aria-label, and data-testid. Locator evidence only helps find an element again; it does not prove that the task was completed.
Parameterize: Replace literals from the first run with template variables—for example, convert test@example.com, the subject, and the body into {recipient}, {subject}, and {content}—while leaving stable actions unchanged. The teaching implementation uses regular expressions and template replacement; a production system may use structured task input or a constrained extraction model.
Define state checks: Add checks before and after actions, such as “the send button is visible” and “the URL after navigation belongs to the target site.” Add a final-state check for the workflow as a whole, such as “the sent-mail list contains the new message” or “the test page’s state value changed as expected.” Successfully executing an action is not the same as successfully completing the task; the final check must read the real page or backend state.
Validate the candidate: A first success produces only a candidate. The system must reset the sandbox account or test site to an independent initial state and replay the candidate in full. It can be published as validated only if all before-action, after-action, and final-state checks pass. If a side-effecting task such as sending mail or placing an order has no safe reset callback, the workflow may be retained as an auditable candidate but must not be validated by repeating the action in a production account.
Match and replay: When a new task arrives, search the formal capability library for a workflow by intent and keywords, extract the current parameters, and execute it directly with Playwright. Replay requires no step-by-step LLM calls, but it must still wait for elements to become available and complete every state check.
Invalidate and relearn: If the target element cannot be found, a state check fails, the API Schema changes, or the final state is wrong, stop subsequent actions immediately, move the old version from the searchable library to the invalid area, and fall back to the full Agent for fresh exploration. Retain the old file for audit and comparison, but never let it continue to match silently.

For an email workflow, the compiled result is not merely “click these buttons in order,” but a small program parameterized by recipient, subject, and body: it checks the compose window and fields before sending, checks the success indicator afterward, and finally confirms that the corresponding message appears in the sent list. In PreAct, such programs delivered an 8.5–13× end-to-end speedup on repeated tasks and required no step-by-step language-model calls during replay. More importantly, process memory needs before-action validation, after-action validation, and independent pre-storage validation. Otherwise, the system can produce a dangerous illusion: replay coverage is 100 percent and every button was clicked, yet one field was empty and the task was never actually completed.

An Agent modifying its own code does not mean that the running process directly overwrites itself. A production system should create a candidate branch from the current stable version, have a Coding Agent generate a minimal patch, and then sequentially run static checks, unit tests, security scans, failure-trajectory replay, and regression tests on old tasks before producing a new version eligible for canary deployment. This turns “self-modification” into an auditable software release process and defines the boundary between Chapters 8 and 5: Chapter 5 provides the capability to modify systems, while this chapter provides a method for self-modification that is triggered by experience and constrained by a validation loop.

Making the patch small is not enough for reliable attribution. Each modification request should also be a falsifiable change contract that records the failure evidence, inferred root cause, responsible Harness component, candidate change, behavior expected to improve, existing behavior that may regress, and tests for both. Agentic Harness Engineering describes this in terms of component-, experience-, and decision-level observability: every editable component has a file-level representation; large collections of trajectories are distilled into evidence that can be inspected at increasing levels of detail; and every edit declares an impact prediction before execution, which the next round of results then tests. A higher score can then be connected to a specific mechanism rather than remaining an uninterpretable trial.

The candidate generator should not receive only failed cases. Self-Harness also supplies successful behavior that must be preserved and records of previously rejected modifications. The former tells the Agent what the repair must not break; the latter prevents it from resubmitting the same failed idea in different words. Failure evidence, success constraints, and prior attempts together define a bounded candidate space and are more useful than indiscriminately loading all source code and raw logs into the modifying Agent.

Tool creation follows the same protocol. Alita presents a case in which an Agent must identify the number mentioned immediately after dinosaurs first appear in a YouTube 360 VR video narrated by the voice actor for Gollum in The Lord of the Rings. After recognizing that it lacks subtitle-reading capability, the Agent finds and tests youtube-transcript-api, wraps it as a new subtitle tool, and extracts the answer 100000000 from the transcript. A new tool enters the capability library only after safety scanning, functional tests, and successful reuse on later tasks. Chapter 4’s proactive tool discovery asks which existing tool fits; Chapter 5 asks how to write a tool; this chapter asks what operational evidence should trigger creation and how a new tool becomes a validated long-term capability.

Encoding Experience in Parameters

Knowledge, instructions, and programs all rest on one premise: the target capability can be expressed relatively completely through external symbols. Yet capabilities such as medical-image understanding, natural speech prosody, removing a formulaic “AI feel” from text, and long-horizon planning are difficult to compress into a few rules or workflows. Such capabilities must be written into model parameters through post-training.

Whether a capability should be parameterized is not determined solely by whether the task is stable over the long term. Domain shifts caused by new imaging equipment may still require LoRA or continual fine-tuning; rapidly changing linguistic styles can also be accommodated through periodic preference training. Stability affects update frequency and cost, but the representational nature of the capability determines its primary medium. Conversely, a long-stable rule for approving transfers should not rely solely on parametric memory; server-side code must still provide deterministic guarantees.

Chapter 7 provided a complete discussion of SFT, distillation, and RL, so this section does not repeat it. For continual evolution, the key is to transform evaluated production trajectories into training data: high-quality demonstrations can be used for SFT, explicit preferences can form paired data, and interactions with reliable environmental rewards can be used for RL. Before training, private information must still be removed, erroneous trajectories filtered out, and an independent regression set retained. After training, the system must check whether general capabilities or safety alignment have been forgotten.

Parameter learning usually works in conjunction with external methods. A medical-imaging model can learn visual representations through parameters, obtain the latest guidelines from a knowledge base, and use code to measure lesions and calculate risk. A natural customer-service tone can be shaped at the distributional level through preference training, while a Prompt specifies the current brand identity and user memory adapts communication to individual preferences. Continual evolution does not mean selecting a single answer from among the four methods, but placing each capability in the medium best suited to expressing and governing it.

From Updating Artifacts to Updating the “Update Method”

The preceding four methods ask where experience is written, but continual evolution has another, orthogonal axis: is the system optimizing the contents of an artifact, or the method used to produce, manage, and validate artifacts? Along this axis, the optimization target can expand from an individual rule or memory → structured context → workflow → Harness code → optimizer code that generates candidate solutions. These are not five new update carriers but five search scales; knowledge, Prompts, Skills, and programs may appear at several of them.

The innermost level changes only artifact content—for example, adding a local rule to a system Prompt after a failed trajectory or adding an exception to an experience document. Such changes have a small blast radius and are easier to attribute and roll back, so they should be the default. Repeatedly asking a model to rewrite an entire Prompt or memory, however, introduces another form of degradation: successive attempts at brevity can gradually erase rare but important details, and interacting constraints can be collapsed into an overgeneral principle. Agentic Context Engineering (ACE) maintains context as a collection of entries with stable identifiers. Generation, reflection, and curation modules propose incremental updates, which deterministic logic merges and deduplicates instead of rewriting an ever-shorter text block each round. It is a concrete research example of this chapter’s earlier principles of minimal diffs and retained provenance.

At the next level, the optimization target is no longer merely what context contains but how context is constructed. Meta Context Engineering (MCE) separates the two into inner and outer loops: the inner loop optimizes the context artifact for the current task under a given management method, while the outer loop uses results from multiple executions and validations to modify the context operations themselves—search, selection, filtering, and formatting. The distinction matters. Editing a retrieval rule changes a content-management mechanism; comparing several retrieval and curation mechanisms and retaining the one with better transfer is learning how to manage context.

The same idea extends to workflows and the entire Harness. AFlow represents workflows composed of multiple LLM calls as code graphs and searches over combinations of nodes and control flow using execution feedback. Meta-Harness has a Coding Agent inspect candidate Harness source, scores, and trajectories to search the code that determines how information is stored, retrieved, and presented. Chapter 5 established code as a general language for expressing Agent system structure. The additional point here is that code, together with its evaluation history, can itself become the object of continual search rather than a one-time output.

Higher levels are not automatically better. Searching for a local rule may require only a few edge cases, whereas searching an entire workflow or Harness faces a much larger candidate space, higher evaluation cost, and harder attribution. A clear, recurring fault localized to one component should first receive an auditable local patch. Only when local changes repeatedly fail to address a cross-component problem, or when the current management method itself becomes the bottleneck, is it worth moving outward to the workflow, Harness, or optimizer. At every level, evaluators, permission boundaries, and held-out tests must remain outside the editable scope—the larger the search space, the more important this trusted root becomes.

Engineering Practice

Zapvol has the write-points for the first three of the four loci, but the automated experience-learning pipeline is not yet closed (see the evolution loop):

Knowledge — the memory system’s feedback type is exactly “corrections or confirmations of how to act,” the write-point for action experience; the offline pipeline that contrasts many trajectories into an experience doc is the gap.
Instructions — the prompt system’s L1 supports DB-sourced agent custom instructions, and Skills support on-demand loading; the System-Prompt-Learning “failure → minimal diff → regression” auto-loop is the gap.
Programs — tools carry deterministic procedure and constraints via ServerToolConfig; a trajectory-triggered self-modification release flow is the gap.
Parameters — model post-training is Chapter 7, out of Zapvol’s current scope.

In other words, Zapvol has the four places to write experience to; the missing layer is the one that decides what to write and ships it safely from a verification loop.

Learning Signals — the evaluation premise behind whether and what to change
The Evolution Loop — candidate verification, gray release, rollback