Route
Cost scales with task difficulty, not with your most conservative default.
Every request has more than one capable model, at very different prices. The LLM Router optimizes your token spend by picking the right one every time: it classifies the task, scores quality against cost, learns from every outcome, and switches only at moments where switching is free.
02 · Decision layer
Gate · Classify · Score · Learn
01 · Dispatch layer
One API · Fallbacks · Allowlists · Keys · Limits · Spend
Two layers, one router: dependable dispatch beneath, an intelligent decision layer on top.
01 · Dispatch
Every provider, one integration
A complete dispatch layer, out of the box: one API across every provider, automatic failover, governed keys and limits, and spend visibility from the first request.
02 · The decision layer
Four moves per request
Gate, classify, score, learn: four moves on every request. The scoring stage is live below, running the real formula with the weights in your hands.
Scoring simulator
Balanced| Model | Q̂ | $ / 1M | Score | Routed |
|---|---|---|---|---|
| nano-3 | 0.52 | $0.40 | 0.616 | |
| mid-3 (routed) | 0.74 | $4.20 | 0.755 | |
| max-3 | 0.93 | $21.00 | 0.744 |
- Routed
- mid-3
- Gap to pay up (w_c / w_q)
- 0.25
- vs always-max
- −80%
score = w_q · Q̂(category) + w_c · (1 − cost̂). That's the router's real scoring rule. Drag the weights, switch the task, watch the choice change.
03 · The boundary rule
Routing is free only at boundaries
Switches land at exactly two moments, the first turn of a session or a formal agent handoff, because those are the moments a switch costs nothing. Four disciplines make that guarantee real.
Cache economics
Every cached token keeps working.
Cached input runs up to 10× cheaper, and boundary discipline preserves the cache through the whole task, so the discount compounds instead of resetting.
Tool continuity
Tool context stays whole.
The model that makes the tool calls is the model that finishes the task: full context, clean results, no retracing.
Clean handoffs
Fresh context, by design.
Every switch lands exactly where a fresh reasoning context begins, so nothing is lost in the handover.
Coherence
One voice per task.
Reasoning style stays consistent from the first token to the last, so your users see one seamless answer.
Precise by design
Boundaries come from your framework's own lifecycle events, never guessed from conversation content. Every boundary decision is visible on the dashboard, live.
04 · Exploration
Routing that keeps getting smarter
Exploration-aware selection keeps re-testing every candidate, so a model that improves (or a new one you add) earns traffic on evidence. Naive greedy routing locks onto one model and stops looking; Route never does, and it wins on quality per dollar at every setting.
| Weights (Q / C) | Greedy | Route | Δ |
|---|---|---|---|
| 0.9 / 0.1 | 2.59 | 3.28 | +27% |
| 0.8 / 0.2 | 2.96 | 5.72 | +93% |
| 0.7 / 0.3 | 7.44 | 12.54 | +69% |
Quality per dollar, higher is better. Route wins at every weight setting, and at the default it spends 53% less doing it. Benchmark: 3,000 queries across two model tiers.
05 · Governed weights
Per-team weights, not one global slider
One global dial can't govern a company. Route's weights are set per team and per deployment, governed like any other policy. Each setting is an explicit, predictable threshold for when extra quality is worth paying for.
Cost-first
- Weights (Q / C)
- 0.7 / 0.3
- Gap to pay up
- 0.43
- Quality per dollar
- 12.54
Batch enrichment, tagging, backfills: tolerant work where the cheap model is almost always right.
Balanced
Default- Weights (Q / C)
- 0.8 / 0.2
- Gap to pay up
- 0.25
- Spend saved
- −53%
Product surfaces with a human reading the output. What a team gets if it sets nothing.
Quality-first
- Weights (Q / C)
- 0.9 / 0.1
- Gap to pay up
- 0.11
- Exploration
- always on
Work where a weak answer costs more than the strong model does. The router pays up on an 11-point edge.
Candidate set
Registered models ∩ the governance allowlist. A blocked model is never scored, so policy runs upstream of the decision.
Budget headroom
w_cost tightens automatically as a team approaches its spend limit, instead of staying static while the budget burns.
Config lock
Routing config can be tightened by policy but never loosened by a caller.
One ledger
Every decision lands in the same spend ledger TokenOps monitors. Route is its acting arm.
06 · Guarantees
What Route holds itself to
Five commitments, each one measurable on the dashboard, so trusting the router never has to mean taking its word for it.