Choosing an Agent Harness for Production

The loop is not the hard part

A useful harness is judged on what it does when things fail. The happy path—send a prompt, receive a tool call, execute it, return the result—is a loop anyone can write in an afternoon. The engineering decision begins when the process crashes after an external action, a tool returns something unexpected, or an upgrade changes which action the model chooses.

The realistic choices are a plain loop you own, a framework harness you run, and a managed harness whose runtime is operated for you. This is narrower than choosing a model. It is also narrower than deciding between an agent framework and a plain tool-calling loop: the question here is which option gives you acceptable failure behaviour without taking control of the parts in which your team has invested.

Anthropic’s December 2024 account of building effective agents says successful implementations often use simple, composable patterns. It also warns that frameworks can obscure prompts and responses, making failures harder to debug. That does not make frameworks a bad choice. It puts the burden on a framework to provide more than convenient loop syntax.

Failure recovery: require evidence, not a feature name

For a plain loop, resumption after a crash does not exist until you design it. The loop must persist enough state to distinguish an unfinished model turn from a completed tool action. A useful checkpoint records the run and step identifiers, the request sent to the tool, its structured result, and whether the result was returned to the model. Side-effecting calls also need the idempotency-key design appropriate to agent actions; otherwise “resume” can mean charging, sending, or deleting twice.

The cost is engineering time and ownership of the recovery path. The benefit is that the restart boundary matches the semantics of your own tools. A plain loop is the wrong answer when interrupted runs must survive immediately and the team cannot build and test that machinery.

A framework harness earns its abstraction if it supplies durable checkpoints and lets you choose where they occur. Test this by killing a run before a tool call, during it, and after the external action succeeds but before the result is recorded. “Persistent memory” or “durable execution” is not enough unless the resumed run avoids both losing the result and repeating the action.

A managed harness may remove operation of the checkpoint store, but management is not proof of correct resumption. Ask what state is durable, what the replay boundary is, how long run data remains available, and whether a failed run can be resumed on a pinned runtime version. If those answers are unavailable, treat recovery as unverified.

Structured tool errors: protect the investment

Tool definitions are where the real investment accumulates: schemas, descriptions, authentication boundaries, validation, error meanings, and the edge cases learned in production. Portability of that layer is worth more than portability of the loop. Rewriting a short control loop is cheap; translating every tool into a harness-specific contract and retesting model behaviour is not.

A plain loop gives full control over the tool contract. Define failures as data the loop can inspect rather than prose the model must interpret: distinguish a retryable failure, an invalid request, a denied action, and an unknown outcome. The cost is that your team must maintain the schema, adapters, and retry rules.

A framework stands well here only if tool definitions remain ordinary data or code that can be adapted without rewriting their meaning. Export one representative tool—including its structured errors—into a second runner before committing. If the framework requires business logic inside decorators, callbacks, or proprietary result objects, the loop has become portable while the valuable part has not.

A managed harness has the same test with an additional boundary: can the identical tool contract run outside the service? If not, migration means rebuilding the tool layer. That can still be acceptable when the tools are few and disposable. It is the wrong trade once the catalogue contains production permissions, failure semantics, and carefully tuned descriptions. Keep the separate permission model for agent tools explicit; a transport schema should not silently decide what an agent may do.

The authors of A Survey on Large Language Model based Autonomous Agents, revised March 2025 organize the field around agent construction, applications, and evaluation. The practical inference is that a loop is only one replaceable part of a larger system. Tool contracts and evaluation records should therefore cross the harness boundary rather than be trapped inside it.

Tracing: capture the decision boundary

A useful trace must let an engineer reconstruct the sequence of model requests, tool calls, structured errors, retries, and checkpoints. A log line saying “step failed” is not tracing. The point is to determine whether the model chose the wrong tool, the harness changed the request, the tool rejected valid input, or recovery repeated a completed action.

With a plain loop, the trace format is yours and so is the work. Record events at the boundaries you control and keep stable identifiers across retries and resumption. This is a good fit when existing observability infrastructure can receive those events. It is a poor fit when the team would otherwise have no searchable run history during an incident.

A framework should expose its trace data rather than only render it in a bundled viewer. Confirm that raw events can be exported and that hidden framework steps are visible. A managed harness should meet the same standard, with enough retained data to support the incident window your team actually uses. If export or retention is not documented, mark it unverified rather than assuming the dashboard is sufficient.

Whatever the option, preserve the evidence needed for agent run replay and debugging. Replay is not merely rerunning the initial prompt; it requires the intermediate inputs, results, errors, and version information that produced the original path.

Prompt ownership: find the hidden policy

A harness that owns the prompt owns your behaviour. Ask early how much of the system prompt is yours: can you see the complete rendered prompt, replace it, version it, and run the previous version after an upgrade?

A plain loop has the strongest standing because the application assembles the prompt. Its cost is responsibility for every instruction and for testing changes. A framework is acceptable when its injected instructions are inspectable and overridable. If it hides tool-use guidance or rewrites messages, debugging stops at the abstraction boundary described by Anthropic.

A managed harness is the weakest choice when it can change hidden instructions independently of your release. Do not assume that managed means hidden, but require a concrete answer. If the full effective prompt cannot be captured with a trace, behaviour cannot be pinned through prompt versioning alone.

Upgrade cadence: improvements arrive with work

Upgrade cadence is a commitment. A fast-moving harness gives you improvements and a maintenance burden in the same release. The relevant question is not whether releases are frequent; it is whether your team can hold a known version, inspect changes, replay representative runs, and keep the old behaviour while a regression is investigated.

A plain loop minimizes harness-driven change, although model and tool changes still need evaluation. A framework can be pinned as a dependency, but each upgrade must be treated as a behaviour change when it alters prompts, tool serialization, retries, or checkpoints. A managed harness needs an explicit version policy or a stable compatibility contract; without one, pinning is unverified.

Martin Fowler’s dated collection of generative-model engineering reports, viewed August 26, 2026, shows the surrounding practices continuing to change across prompts, coding agents, evaluation, and harness engineering. That makes an upgrade policy part of the purchase, not administrative detail. Set release thresholds for agent evaluation regressions before adopting a cadence you do not control.

Which to pick when

Pick a plain loop when you have a small platform team, need full system-prompt control, and expect the tool catalogue to outlive the runner. Accept that you must build resumption, structured errors, tracing, and versioned behaviour yourself.

Pick a framework harness when it passes crash tests, exports complete traces, preserves portable tool contracts, exposes its effective prompt, and can be pinned. It is the practical middle choice when those capabilities remove work your team would otherwise have to own.

Pick a managed harness when operating durable runs and trace storage is the immediate constraint, and when its prompt visibility, export path, recovery semantics, and version policy are documented well enough for your release process. Do not pick it merely to avoid writing the loop. If any of those contracts is hidden, choose the framework or plain loop instead.