Frontier or mid-tier models, per task class

The decision most teams get wrong is the granularity, not the model

Most teams pick a model once. Someone runs a bake-off, a tier wins, it goes in the config, and every call the product makes from then on goes to it. That decision is wrong in both directions at the same time. Model selection made once per team rather than per task class overpays on the easy majority of calls and underperforms on the hard minority — and because the two errors are invisible to each other, neither shows up in a review. The spend looks like a fixed cost of doing business, and the failures look like prompt problems.

The useful unit is the task class: a set of calls that share an output shape and a failure mode. Classifying a support ticket is one class. Deciding how to break a migration into steps is another. They have nothing in common except that they both go through the same SDK, and treating them as one decision is what produces the double error.

What each tier is buying you

Providers ship models in tiers that trade capability against price and latency; Anthropic’s model overview and comparison table is the current statement of what its own tiers cost and where each is positioned, and the other major providers publish the equivalent. The tiering is not marketing invention — the tiers differ in ways you can measure — but the size of the gap is entirely task-dependent, and that is the part a single team-wide choice throws away.

So the criterion is not “how good is this model”. It is “on this class of task, does the gap between tiers exceed what the gap costs me”.

Where mid-tier holds: small, checkable output spaces

Classification, extraction, and routing are the task classes where a mid-tier model most often matches a frontier one, because the output space is small and checkable. Those two properties are the mechanism, and they are worth separating.

Small means the model is choosing from a bounded set — one of eleven ticket categories, one of four downstream handlers, a fixed set of fields lifted out of a document. There is less room for a more capable model to be more capable in. Both tiers converge on the same handful of answers because there are only a handful of answers.

Checkable means you can tell, cheaply and mechanically, whether a given output was right. A label either matches the human label or it doesn’t. An extracted date either parses or it doesn’t. That makes the residual error rate measurable rather than a matter of taste, so you can see the gap between tiers as a number instead of arguing about which output “reads better”.

The cost side is where this lands. These classes are usually the high-volume ones — every ticket, every document, every inbound request — so they dominate the bill. Sending them to a frontier model is where the overpaying happens, and the cost and latency attribution work that tells you which classes actually dominate your spend is a prerequisite for knowing whether this is worth doing at all. Mid-tier models are also faster, which matters when the call sits in a request path rather than a background job.

Where the gap persists: open-ended reasoning, long-horizon planning, first-run code

Open-ended reasoning, long-horizon planning, and code that must run first time are where the gap persists and where the cheaper choice costs more in rework. The mechanism is the inverse of the one above: the output space is large, and correctness is expensive to check.

When an agent plans a ten-step change, an error in step two does not surface as a wrong label. It surfaces as six wasted steps, a half-applied migration, and an engineer reconstructing what happened. Errors compound along the horizon rather than staying local, so a small per-step quality difference turns into a large per-run difference. Code that must run first time has the same shape: the failure is not “slightly worse code”, it is a failed build, a re-prompt, a review cycle, and the engineer’s attention pulled back to something they thought was done.

That is the honest accounting of “cheaper”. The token bill goes down and the rework bill goes up, and the rework bill is paid in engineering time, which is the more expensive currency. On these classes the frontier model is usually the cheaper choice once you count both.

Be careful about extending this into a claim about whether the tools make developers faster overall. They might not; the field reports and studies disagree with each other, and Martin Fowler’s team has been publishing running field notes on generative AI in software delivery that document exactly that disagreement rather than resolving it. The narrow claim here is about rework within a task class, not about aggregate productivity.

Task class	Output space	Tier that usually wins	Why
Classification	Small, bounded	Mid-tier	Both tiers converge; volume dominates cost
Extraction	Small, schema-checked	Mid-tier	Errors are mechanically detectable and rare
Routing	Small, in request path	Mid-tier	Latency matters; the decision is cheap to verify
Open-ended reasoning	Large, taste-dependent	Frontier	Quality gap survives; checking is expensive
Long-horizon planning	Large, compounding	Frontier	Early errors invalidate later steps
First-run code	Large, binary pass/fail	Frontier	Rework costs engineering time, not tokens

The criterion nobody budgets for

Here is the part that decides whether this is worth attempting. A per-task-class decision needs a per-task-class evaluation, so the cost of choosing well is an eval suite rather than a spreadsheet. You cannot reason your way to “mid-tier is fine for our extraction” from a pricing page. You need a held-out set of your own inputs, per class, and a way to score outputs on each tier.

For the small-checkable classes that suite is cheap: a golden set of labelled examples and an exact-match comparison. For the open-ended classes it is not, which is why model-as-judge exists — the LLM-as-a-judge and MT-Bench paper is the standard reference, and it reports that a strong model judge agrees with human raters at roughly the rate two human raters agree with each other, while also naming the biases (position, verbosity, self-preference) that make that agreement conditional rather than free. Which method fits which class is its own decision, covered in golden sets versus model-as-judge, and the mechanics of running either against a non-deterministic system in CI are the actual engineering cost.

Budget it honestly. If you will not build the eval suite, do not split the decision by task class — you will be swapping a defensible single choice for an undefended set of guesses. The routing layer that dispatches classes to different models is the easy part, and multi-model routing covers the implementation; the eval suite is what makes the routing table mean anything.

The decision expires

Prices and capabilities change on a cadence of months, which means a selection decision needs a stated review date rather than being treated as settled. A mid-tier model shipped this quarter may clear a bar that only the frontier tier cleared last quarter, and prices move in the same window. A routing table written a year ago and never revisited is not a decision; it is a fossil.

Write the review date into the same file as the routing table, and re-run the eval suite on that date. That is the practical reason the suite is worth building: it is not a one-off procurement exercise, it is the thing that makes the next re-evaluation cost a morning instead of a month.

Which to pick when

• Pick mid-tier for classification, extraction, and routing — small output spaces, mechanically checkable outputs, high volume. This is where a single frontier-tier default overpays, and it is usually the majority of your calls.
• Pick frontier for open-ended reasoning, long-horizon planning, and code expected to run first time. The gap persists on these classes and the cheaper model pays it back as rework in engineering time.
• Pick one model for everything if you are not going to build a per-class eval suite. An undefended split is worse than a defended default. Pick frontier if failures are expensive, mid-tier if volume is what’s hurting, and say which you chose and why.
• Split by class, but only after you have a held-out set per class and a scoring method you trust — exact-match for the checkable classes, judged evaluation with its biases acknowledged for the rest. That suite, not the routing code, is the real cost of choosing well.

Whatever you pick, put a review date on it. The pricing and the capability ordering that justified today’s choice have a shelf life measured in months.