Cheap and fast: choosing budget LLMs for high-volume work
A 3-step workflow to find the cheapest model that still clears your quality bar — measured as cost per acceptable output.
Not every call needs a frontier model. Classification, extraction, routing, summarization, and the inner loop of agents are all high-volume jobs where a fast, cheap model wins — if it clears your quality bar. Here's how we choose.
The trap: optimizing tokens instead of outcomes
It is tempting to sort the model list by price and pick the cheapest row. That backfires when the cheap model fails 15% of the time and you bolt on retries, a judge, and a fallback. The real metric is cost per acceptable output.
The honest way to find it is to measure on your data, not to trust a vibe.
A 3-step budget workflow
- Estimate the per-request cost across candidates with the Token & Cost Estimator. It pulls live gateway prices and your token counts.
- Measure quality by running the same prompt over a representative dataset and scoring each output with Bulk Evals. Now you have a pass-rate per model.
- Divide: cost per request ÷ pass-rate = cost per acceptable output. Rank by that.
| Workload | Start with | Why |
|---|---|---|
| Sentiment / intent classification | Fast tier | Short outputs, easy to verify, latency-sensitive |
| Document extraction to JSON | Fast → mid if schema is complex | Cheap models drift on nested schemas |
| Summarization at scale | Fast tier | Quality plateaus quickly; pay for tokens not brains |
| Agent inner loop (tool calls) | Mid tier | Tool reliability matters more than raw price |
| Final user-facing answer | Mid → frontier | Where quality is most visible |
Make the cheap model good enough
Before you reach for a bigger model, try squeezing more out of the small one:
- Optimize the prompt. A tuned prompt often closes the gap with a model one tier up. The Prompt Optimizer does this automatically against your eval.
- Constrain the output. Ask for JSON, a single label, or a bounded format — cheap models follow tight specs better.
- Route dynamically. Use
app/auto-fastfor the easy 80% and escalate only the hard 20%.
The cheapest production setup is usually a fast model with a good prompt and a cheap judge — not a frontier model everywhere.
One balance for all of it
Every tool above bills from the same prepaid credit balance at the published app.nz rates, with no per-seat fees. Top up once and run optimizer sweeps, evals, and comparisons from the same wallet.