What an Agent Should Persist Between Steps and Runs

What persisted state is

An agent’s persisted state is whatever it writes outside its own context window so that a later step, or a later run, can read it back and carry on. It comes in three kinds with three different lifetimes — conversation history, working state for the current task, and durable knowledge that outlives the run — and most storage mistakes come from handling one of them with the rules that suit another.

Three things that get conflated

The three are easy to run together because, in a naive agent, they all live in the same place: the transcript. Every message, every tool call, every result and every conclusion is appended to one growing list that is replayed to the model on each step. That works until the first time the list is too long, the process dies, or a second run needs what the first one learned — and each of those failures is a different kind of state escaping from a container that was never designed to hold it.

Conversation history is the record of what was said and done: prompts, model turns, tool invocations, tool results. Its lifetime is one run. It is append-only by nature and grows with every step.

Working state is what the agent has established about the task so far — which subgoals are done, which files or records it has touched, what values it has looked up, which decisions it has made and why, and what it is blocked on. Its lifetime is the task, which may span many steps, an interruption, a compaction of the context window, or a restart on another host. It changes as the task progresses, and later entries supersede earlier ones.

Durable knowledge is what should outlive the run: the user’s stated preferences, facts about the environment that were expensive to discover, corrections a person made that should not have to be made twice. Its lifetime is indefinite, which is exactly the property that makes it dangerous.

State type	Useful lifetime	Update pattern	Primary risk
Conversation history	One run	Append, then compact or discard	Re-reading dead detail at every step
Working state	One task, across restarts	Rewrite at verified step boundaries	A stale record misleading the next step
Durable knowledge	Across tasks and runs	Supersede, recheck, and evict	Old or poisoned memory being trusted as current

The 2023 survey of large-language-model-based autonomous agents draws the memory module along similar lines, distinguishing the short-term memory an agent holds in context from long-term memory kept in an external store and retrieved on demand. The distinction that survey does not force, and that this page does, is the middle one: working state is neither the transcript nor the long-term store, and it needs its own home.

Conversation history: the most expensive and the least valuable

Of the three, conversation history costs the most and is worth the least, and both properties follow from the same mechanism. It grows without bound — every step appends and nothing removes — and the model reads all of it on every step, so the token cost of step n is proportional to everything that happened in steps one through n minus one. A run of a hundred tool calls pays for the first tool result a hundred times.

Meanwhile most of what it holds is superseded by its own conclusions. The forty lines of directory listing that led the agent to conclude that the config lives in settings/ are worthless once that conclusion is written down; the three failed attempts at a command are worthless once the fourth succeeds; the reasoning that produced a decision is worth less than the decision. Anthropic’s write-up on building effective agents describes the agent loop as one where the model gains ground truth from the environment at each step through tool results — and the corollary is that most of that ground truth is instrumental, consumed on the way to a conclusion and then dead weight.

This is why the practical answer for history is to treat it as disposable and design so that its loss costs nothing. Summarise it, truncate it, drop the middle, keep the last few turns for local coherence — the specific compaction tactics are covered under keeping an agent’s context useful as a run gets long — but do it on the assumption that anything the run actually needs has already been moved somewhere else. If compaction loses something the agent needed, the failure is not the compaction; it is that a piece of working state was living in the transcript.

Two things history is good for: audit and replay. A complete record of what happened is what a person reads after the fact to work out why an agent did what it did. That is a reason to log the transcript durably, not a reason to feed it back to the model.

Working state belongs in a structure the agent writes deliberately

Working state should live in something the agent writes on purpose — a file, a row, a document, a small structured record — not in the transcript as a side effect of talking. The mechanism matters: the transcript is what gets compacted, so anything that lives only there is at the mercy of whatever summarisation happens next. A file the agent wrote survives compaction untouched, survives the process dying, and can be read by the next step, the next run, or a person, without replaying anything.

Deliberate means the agent chooses to write it, at known points, with a known shape. In practice that is a small set of named things: the goal as currently understood, the plan or the list of subgoals with their status, decisions taken and the reason for each, values discovered that a later step will need (an ID, a path, a version number), and what is currently blocked and on what. The shape should be stable enough that a step can read it without interpretation and small enough that rewriting the whole thing each time is cheaper than a diff.

The right granularity is the step boundary. Write after every step that changed what is known; read at the start of every step. An agent that only writes at the end of a run has working state that vanishes with any interruption, and one that writes on every token has turned its state file into a second transcript. Both are the same error in different directions.

The cost is a little more prompting and one more place things can be stale. If a step updates a resource but does not update the record, the next step believes the old value. That is a real failure mode, and it is why the record should hold facts the agent has verified rather than facts it intends to make true. deployed: true should be written after the deploy returned success, not before it was attempted.

The pay-off is that the record becomes the unit of hand-off. When a task moves from one agent to another, or from an agent to a person and back, what moves is the working-state record, and the receiver starts from it rather than from a replayed transcript. The mechanics of that transfer are covered under handing work between agents and the systems around them; the precondition is that the state exists as a thing that can be handed.

Durable knowledge needs an eviction story from the start

The instinct with long-lived memory is to add and never remove. Every run learns something; the store grows; recall gets richer. That is true for a few weeks. After that, memory that only grows becomes retrieval noise, and then it becomes wrong without anyone noticing.

The mechanism has two stages. First, noise: retrieval returns the top few matches for a query, and as the store grows, the chance that the top matches are the ones that matter falls. Ten near-duplicate notes about a preference crowd out the one note about a constraint. Second, staleness: a fact written eight months ago about a service’s limit, an API’s shape, or a person’s role stays retrievable at the same confidence as one written yesterday, and nothing in a plain key-value or vector store distinguishes them. The agent reads it, trusts it, acts on it, and the error surfaces somewhere downstream with no obvious link back to the memory that caused it.

Eviction has to be designed at the same time as insertion, because retrofitting it means deciding what to delete from a store nobody understands any more. The pieces:

• Provenance on every entry. Who or what wrote it, from what evidence, on what date. Without a date there is no way to say what is old.
• A lifetime, or a recheck interval. Facts about the world get a date after which they are treated as unverified until confirmed. Preferences a person stated explicitly can live longer than inferences the agent made from behaviour.
• Supersession, not accumulation. A new fact about the same subject replaces the old one rather than sitting beside it. That needs entries keyed by subject, which is a design choice that has to be made before the first write.
• A cap and a review. A store that has a maximum size forces the question of what to drop, and a store that a person can read end-to-end in ten minutes is one that will actually be reviewed.
• Deletion as a first-class operation. If there is no command or path by which an entry is removed, entries will not be removed.

The cost is that the agent sometimes has to rediscover something it once knew. That is cheaper than confidently acting on something that stopped being true.

Anything persisted can be poisoned

A memory store is an input. Every fact in it was written by something — a previous run of the agent, a tool result the agent copied, a person — and every fact in it is read by the next run as if it were trusted. A fact written by one run and read as trusted by the next is prompt injection with a delay: the attacker does not need to be in the conversation, only to have got something into the store at any point in the past.

The route in is usually a tool result. An agent reads a web page, a ticket, an email, a file in a repository, and some of that content is written into working state or durable memory as though the agent had established it. If the content was crafted, the store now holds an instruction dressed as a fact, and it will be retrieved and obeyed in a later run whose author never saw the source. The OWASP Top 10 for large-language-model applications puts prompt injection at the top of its list and treats content the model retrieves from stores as one of the routes it arrives by, alongside the direct conversational route.

What follows for persisted state:

• Record provenance and use it. An entry that came from an untrusted source is retrieved with that label attached, and the model is told to treat it as a claim from that source, not as an established fact.
• Separate what the agent concluded from what it copied. A quotation from a tool result and a decision the agent made are different things and should not sit in the same field.
• Gate durable writes harder than working-state writes. Working state expires with the task; the blast radius of a bad entry is one run. Durable knowledge is read by every future run, so writes to it are a good candidate for a person to confirm — the same reasoning that decides which actions need approval under permission models for what an agent may do without asking.
• Never persist instructions, only facts. A memory entry that says always do X is a standing order the next run will follow. Store the observation that led to it and let each run decide.
• Make the store diffable. If persisted state is text under version control, a poisoned entry is a line in a diff someone can see. If it is an opaque embedding index, it is not.

State that determines behaviour should be readable by a person

Debugging an agent means asking what it thought was true when it did the thing it did. If the answer lives in a plain file the person can open — a markdown record, a JSON document, a row with named columns — the question takes a minute. If it lives only in an embedding index, a binary blob, or a transcript that was compacted three steps before the failure, the question is unanswerable, and the debugger is reduced to rerunning and hoping.

Human-readable does not mean unstructured. A YAML or JSON file with stable keys is both machine-parseable and readable in an editor. What it rules out is state whose only representation is one the model can consume and a person cannot: vector stores with no plain-text mirror, serialized objects, or state that exists only as the model’s inferred summary of a long conversation.

The rule applies with most force to state that determines behaviour — the goal, the plan, the current beliefs the agent is acting on. A cache of tool results can be opaque; the file that says what the agent is trying to do and why cannot. This is also the precondition for after-the-fact analysis: reconstructing what an agent did starts from the state records and the transcript log, and if the state records are not readable, the reconstruction has one leg.

The cost is discipline: it is easier to let a framework serialize whatever it holds than to decide what the record contains and keep it in a form a person can read. The alternative cost is a 3 a.m. incident with no way to know what the agent believed.

The test: resume on a different machine

The check that catches most of the above at once is this. Take a run part-way through a task. Kill it. Start a fresh process on a different machine, with access to the persisted store and nothing else — no transcript, no context, no local files that were not written to the store on purpose. Can it pick up where the first left off?

If yes, the working state is genuinely in the store. If no, the state was in the transcript, or in the local filesystem by accident, or in the head of the person who was watching, and the store is a comforting fiction.

The test is precise about what it demands:

• Every value a later step needs was written down, not merely observed. If the agent looked up an ID in step three and used it in step nine, the ID is in the record.
• The record says what is done, so the fresh process does not redo completed work — and does not skip work it only intended to do.
• Nothing depends on the machine. Temp files, environment variables set at runtime, and credentials that were provisioned mid-run are all things a different host will not have unless they were persisted, or unless the design accepts re-provisioning them.
• The store is reachable from where the agent runs, which is a deployment question as much as a design one; a state file on one developer’s laptop fails the test by construction.

Run it early, when the store is small and the failure is cheap to fix. Run it again whenever a new kind of state is added, because the usual regression is a new value that some step started relying on without anyone writing it down.

What to check next

A reader applying this will next want three things this page only points at: how aggressively the transcript can be compacted once working state has a home of its own; where in the loop a person confirms a write to durable knowledge, and how that confirmation avoids stalling the run; and how a run’s state records and its transcript log are combined to reconstruct what happened when something goes wrong. Each is its own subject with its own trade-offs, and each assumes the separation described here has already been made.