Agent frameworks vs a plain tool-calling loop

What each option actually is

A plain tool-calling loop is a while statement around a model call, a tool dispatch table and a message list. The model returns either an answer or a tool request; the loop looks the tool up, runs it, appends the result to the message list, and calls the model again. Add a stop condition and a turn cap and you have an agent. For a single-purpose agent — one that files a ticket, reconciles an invoice, answers questions over one corpus — it is usually under 200 lines, and the important property is not the line count but that the team fully understands every one of them. There is nothing in the call path that someone on the team did not write.

An agent framework is a library that owns that loop for you and surrounds it with machinery. It is worth being precise about what the machinery is, because the pitch is usually about the loop and the loop is the least of it. What a framework actually supplies is rarely the loop: it is retries with backoff, tracing of every model and tool call, streaming of partial output to a client, structured tool schemas derived from typed function signatures, state persistence between turns and across process restarts, and a migration path when the provider changes its API shape. Every one of those is a real thing you would otherwise write. None of them is the while statement.

That framing comes from the people who ship the models. Anthropic’s guide to building effective agents recommends starting with direct model calls, adding a framework only once you understand what it is doing on your behalf, and warns that the abstraction can hide the prompts and responses you most need to see. The academic literature pulls the other way: the 2023 survey of LLM-based autonomous agents decomposes an agent into profile, memory, planning and action modules, and that decomposition maps neatly onto a framework’s class hierarchy. Part of the pull towards frameworks is that they look like the diagram in the paper. The loop looks like nothing, which is the point.

The rest of this page weighs the two against the criteria that actually decide it. There are six, and they do not all point the same way.

Criterion 1: what you are paying for

Start by listing what the loop does not give you, because that list is the framework’s whole case.

A loop with no retry logic dies on the first 429 or the first malformed tool call. A loop with no tracing gives you a log line per turn if you remembered to add one, and nothing that correlates a slow request with the tool call that made it slow — the subject of attributing agent cost and latency to the work that caused it, which is much easier when the plumbing records spans for you. A loop with no streaming makes the user wait for the whole answer. A loop with hand-written JSON schemas for its tools drifts from the functions they describe the first time someone adds a parameter. A loop with state in a local variable loses it when the process does. And a loop written against one provider’s message format is coupled to that format; when the shape changes, the diff is yours.

Each of these is a mechanism you can add to the loop yourself. Retries are twenty lines. Deriving a JSON schema from a typed function is a small helper in most languages. Tracing is a wrapper around two functions. The honest accounting is that a single-purpose agent needs perhaps two of the six, and two of six is an afternoon. The framework’s advantage on this criterion is real but bounded: it hands you all six at once, some of which you will not use, in exchange for a dependency whose internals you did not write.

Where the framework’s standing improves sharply is the last item, the migration path. Provider APIs do change shape — new content-block types, new tool-result formats, new streaming events — and a framework that tracks them means the change lands as a version bump rather than a rewrite. If your loop talks to one provider and you are content to follow its changes yourself, that is a cost you can carry. If you route between several models, as the page on routing requests between models covers, the framework’s normalisation layer starts to earn its keep, because you would otherwise be maintaining that normalisation yourself.

Criterion 2: the overhead nobody prices

Framework overhead is usually discussed as dependency weight: the transitive install, the version constraints, the security surface. That is real but it is the smaller half.

The larger half is paid in tokens. Most frameworks inject their own scaffolding prompt ahead of the caller’s — instructions on how to format tool calls, how to reason, how to signal completion — and they do it on every turn, because each turn is a fresh model call carrying the whole conversation. On a ten-turn agent run that scaffolding is sent ten times. It sits in the context window ahead of your system prompt, which means your instructions are no longer the first thing the model reads, and it is billed at input-token rates every time.

The plain loop’s standing here is simply that it has no scaffolding unless you write it. The system prompt is yours, first, and exactly as long as you made it. You can measure what a run costs and every token in the bill is one you put there. A framework’s overhead can be small per turn and still be the largest single line item across a fleet, precisely because it is multiplied by turns and by agents and never appears in the framework’s own accounting.

The condition that flips this criterion: if the framework’s scaffolding is doing work you would otherwise write into your own prompt — a well-tested tool-use preamble, say — the tokens are not overhead, they are your prompt written by someone else. Check what it sends. If you cannot see what it sends, that is a finding in itself, and it belongs under criterion 5.

Criterion 3: switching cost runs the wrong way

Intuition says a plain loop is the risky choice because you will outgrow it and have to migrate. The intuition is backwards.

Moving from a loop to a framework is mechanical. The loop’s pieces — the message list, the dispatch table, the tool functions — are exactly the pieces every framework expects you to hand it. You register the tools, pass the system prompt, delete the while. A day, usually less, and the tool functions themselves do not change.

Moving off a framework means re-implementing whichever of its features the product quietly came to depend on. Not the ones you chose — those you know about — but the ones that were on by default: the retry policy that was masking a flaky tool, the state store that three other services started reading, the streaming format the front end was parsed against, the tracing that the on-call runbook links to. You discover the dependency list by removing the framework and seeing what breaks, which is the worst possible way to discover it.

So the reversible choice is the loop. Starting with a framework is not wrong, but it should be understood as the choice that is harder to unmake, not the safe default. If you are uncertain, uncertainty argues for the loop, because the loop is the option you can leave cheaply.

Criterion 4: the three requirements that end the argument

Three requirements most reliably make a plain loop the wrong answer, and they share a cause: each one is a distributed-systems problem rather than a control-flow one. A while loop is a control-flow construct. It can be extended to handle almost anything that stays inside one process and one run. It cannot be extended into a distributed system without becoming one, at which point it is no longer under 200 lines and no longer something the team fully understands.

Multi-agent orchestration. One agent handing work to another, waiting on it, merging results, handling the case where the second agent fails halfway. This is message passing, supervision and partial failure — the standard distributed-systems set — and a framework that has already made choices about them is worth more than the choices you would make under deadline. A single-purpose agent by definition does not have this requirement, which is why the loop suits it.

Durable resumption. An agent run that must survive a process restart, a deploy or a crash and continue from the turn it reached rather than the beginning. This needs the message list, the pending tool call and the tool’s side effects to be checkpointed atomically, and replayed without double-executing anything. It is the subject of durable execution for long-running agent workflows, and the short version is that a loop with state in memory cannot have it, and a loop with state in a database has become a workflow engine you wrote yourself. It also interacts with where the agent runs: the trade-off in serverless functions versus long-running hosts for agent workloads is largely a question of who owns resumption.

Human-in-the-loop approval. The agent proposes an action, a person approves or rejects it, possibly hours later, possibly from a different device, and the run continues from exactly that point. Structurally this is durable resumption with an external signal as the resume trigger, plus an audit record of who approved what. Every hard part of the previous item, plus authorisation.

If your agent has none of these three, the loop is a serious candidate and probably the right one. If it has one, the framework — or a workflow engine, which is the part of the framework you actually need — is very likely correct, and the choice becomes which one. If it has all three, the plain loop is not a real option.

Criterion 5: debuggability

This is the argument that survives contact with production, and it favours the loop.

Agents fail in ways that need reading. Not a crash with a clean exception, but a run that took forty turns to do a five-turn job, or called the same tool eleven times with slightly different arguments, or produced a plausible answer from a tool result that was actually an error message. Diagnosing that means reading the transcript, seeing exactly what the model was sent on turn seven, and seeing exactly what your code did with what came back. A stack trace through code the team wrote beats one through a framework’s abstraction, because the person on call can read the former without first learning the framework’s internal model of an agent step. For a running field record of what teams hit when they put these tools into delivery work, Martin Fowler’s collected memos on generative AI in software delivery is one account worth reading alongside your own incident log.

The loop’s standing on debuggability is close to ideal: the message list is a variable you can print, the dispatch table is a dictionary you can inspect, and there is one place the model is called. The framework’s standing depends on how much it hides. The best frameworks expose the raw request and response and make their scaffolding inspectable; the worst wrap the model call in three layers of callback and log a summary. The Anthropic guide’s warning about frameworks obscuring prompts is exactly this criterion. When evaluating a framework, the test is: from a failed run in production, how many steps to the exact bytes sent to the model on the turn that went wrong? If the answer is more than one, you are paying in debuggability for whatever else it gives you.

There is a fair counter. Framework tracing, when it is good, gives you the transcript view for free, with timing and token counts per span, which a loop only has if someone wrote it. So the honest comparison is loop-plus-your-own-tracing against framework-with-its-tracing. The loop wins when your tracing exists; the framework wins when it does not. That is a statement about your team’s discipline, and worth being truthful about.

Criterion 6: how many agents you run

A framework’s value rises with the number of distinct agents a team runs, because the parts worth having are exactly the parts nobody wants to write twice.

Retries, tracing, streaming, schema derivation, persistence, provider normalisation: for one agent, each is an afternoon and the total is a couple of days. For six agents owned by three teams, writing them six times is waste, and writing them once as an internal library is — a framework. A small, understood, in-house one, which is often the right answer, but a framework nonetheless. At that scale the question stops being loop versus framework and becomes which framework: the one you build from the parts your loops already have in common, or the one you install.

Which pushes the calculation back to the earlier criteria. The in-house version keeps debuggability and token overhead under your control and has no scaffolding prompt you did not write. The installed version has more features, tracks provider changes for you, and carries the switching cost of criterion 3. Neither is free. What is clearly wrong at six agents is six independent loops each with its own retry logic, and what is clearly wrong at one agent is a full framework standing behind 150 lines of business logic.

For teams whose agents share a provider and hit the same quota, there is a practical reason to consolidate early: the retry and backoff behaviour discussed in agent execution under provider rate limits and concurrency caps is one of the things you most want implemented once and identically, because six agents with six different backoff policies against one rate limit is a fleet that starves itself.

Where each option stands

Criterion	Plain loop	Framework
Retries, tracing, streaming, schemas, persistence, provider migration	Write what you need; usually two of six	All six, including ones you will not use
Token overhead	None you did not write	Scaffolding prompt on every turn
Switching cost	Cheap to leave for a framework	Expensive to leave; hidden dependencies
Multi-agent, durable resumption, human approval	Wrong answer once any is required	Where it earns its place
Debuggability	Stack trace through your code	Depends on how much it hides
Many distinct agents	Six copies of the same plumbing	The plumbing written once

Which to pick when

Pick the plain loop if you are building one agent with one job, it runs to completion inside a single process, nobody has to approve its actions mid-run, and the team wants to be able to read every line in the call path. That describes most first agents and a large share of second ones. Write your own retry and your own tracing; both are short, and having written them you will know what a framework is offering when you look at one.

Pick a framework, or a workflow engine, as soon as any of the three requirements appears — orchestration across agents, resumption after a crash or deploy, or a human approval gate. Do not try to bolt them onto the loop; the loop stops being small the moment you do, and you end up with an under-tested framework you also have to maintain. When you evaluate candidates, put debuggability first: from a failed production run, how quickly can you see the exact bytes sent to the model on the failing turn? Then check what scaffolding it prepends and how much of it you would have written anyway.

Pick a framework — possibly your own thin one extracted from the loops you already have — once you are running enough distinct agents that the plumbing is being written for the third time. The parts worth having are the parts nobody wants to write twice; three copies is the signal.

And if you are undecided, decide for the loop. The switching cost runs in its favour: a loop becomes a framework in a day, and a framework becomes a loop only after you have found out what it was quietly doing for you.