Model per step: why AI agents need more than one model

Most teams that start with coding agents pick the best model they can get and run everything on it. It feels like the safe choice. In practice it is the expensive one, and often not the better one either. The question is not which model is best, but which model is best for this step.

What a model per step means

An agent workflow is not one task. On the way from ticket to pull request there are a dozen smaller jobs: read the ticket and decide whether it is clear enough, plan the change, write the code, test it against the running product, review it, write the handover. Each of these jobs has a different shape. Some are judgment calls with three possible answers. Some need to hold an entire codebase in mind. Some are mostly typing.

A model per step means each of these jobs gets the model that fits it. In a playbook, every step already has its own role, its own input and its own gate. Adding its own model is the natural next move: the triage step runs on a small, fast model, the architecture plan on the strongest one available, the build step on a strong coding model, the check on a model that is good at reading and comparing.

The comparison from human teams holds again. Nobody asks the principal engineer to sort the inbox, and nobody asks the newest hire to design the data model. Good teams assign work by what it needs, not by who is most impressive.

Why the result gets better

The price spread is huge. The list prices of a single provider’s current models span two orders of magnitude. At Anthropic, Haiku 5.5 costs $0.10 per million input tokens and Fable 5.1 costs $10. Across providers the gap is wider still. A step that runs a hundred times a day on the wrong tier is where an agent budget quietly disappears.

Small decisions do not need deep thought. Is this ticket a bug or a feature? Does the diff touch the payment module? Is the test output a pass? These are classification questions. A small model answers them in a fraction of the time, and in a workflow with many such questions the waiting time adds up as much as the cost.

Strong models go where they count. The budget that is not spent on triage is free for the steps where quality decides everything: the plan that sets the direction, the review that decides whether code reaches production. The point of a model per step is not to save on every step. It is to spend where it matters.

Smaller context, sharper answers. A step with its own model also has its own context. Chroma tested 18 language models in 2025 and found that their reliability drops as the input grows, even on simple tasks. A model that only sees what its step needs answers more reliably than one that carries the whole run along.

Three points these comparisons usually miss

A failed step costs what its model costs. When a gate stops a step, the step runs again. On a small model, a repeat costs fractions of a cent. On the most expensive model, retries on routine steps are what turns a predictable bill into a surprise. Splitting by model keeps the cost of mistakes small where mistakes are frequent.

Swapping becomes a configuration change. New models arrive every few weeks. When the whole workflow runs on one model, switching is a migration with a risk on every step at once. When each step carries its own model, you swap it on one step, watch that step’s gate for a week and move on. The gate is the test that tells you whether the new model holds.

The research has pointed this way for a while. FrugalGPT from Stanford showed in 2023 that a cascade of models could match the best single model of its time at up to 98 percent lower cost. RouteLLM from LMSYS reported in 2024 that routing between a strong and a weak model kept 95 percent of the strong model’s quality at more than 85 percent lower cost on one benchmark. Both route single requests. A playbook goes one step further, because it knows in advance what each step is for.

Why one strong model for everything is not the safe choice

The case for one model is simplicity. One contract, one set of quirks to learn, one prompt style, and the reassurance that every step gets the best there is. That argument is serious. A badly chosen small model on a step that needs judgment does real damage, and a setup with five models is more to keep track of than a setup with one.

The answer is that the safety comes from the gates, not from the model. A strong model on every step still makes mistakes, it just makes them more expensively and you notice them later. A small model behind a gate that checks its output is safer than a large model without one. And the complexity of several models only exists if each step has to be configured by hand. In a playbook the model is one field of the step, next to its role and its gate.

What we learned along the way

The steps where a smaller model does well are rarely the ones you would guess first. Classification and routing are obvious. Less obvious: summarizing a run into a handover, or checking a pull request against the acceptance criteria of its ticket. Both look like they need a strong model and often do not, as long as the input is clean.

The reverse holds too. The step that most often deserves the strongest model is planning, not coding. A good plan makes the build step shorter and lets a mid tier model write the code. A weak plan turns the most capable coding model into an expensive way to build the wrong thing.

Where to start

Do not redesign the whole workflow at once. Pick one high volume step that ends in a short, checkable answer, triage is usually the best candidate, and run a small model on it next to the current one for a week. Compare the answers at the gate. If the small model holds, switch that step and move on to the next one.

At Kadmo, specialized agents work through playbooks in which every step carries its own model, chosen from any provider and swappable at any time. Because every step ends with a gate, you see right away whether a cheaper model holds. If you want to see what this could look like for your team, we are happy to show you.

Sources

• https://claude.com/pricing
• https://www.trychroma.com/research/context-rot
• https://arxiv.org/abs/2305.05176
• https://lmsys.org/blog/2024-07-01-routellm/