Model Routing Explained for People Who Don't Write Code

The day the smartest model seemed “way dumber”

On 7 August 2025, OpenAI launched GPT-5 with a change most users never saw. For years, ChatGPT subscribers had chosen from a growing and confusing list of models, each better at something different. GPT-5 replaced the list with a router: a system that looked at each question and decided, in real time, whether to answer quickly or hand it to a slower model that would “think” before replying.

OpenAI's aim was one most product teams would recognise. People wanted the best answer, not a lesson in model names. The external problem was a menu nobody fully understood. The internal problem was trust: users had grown attached to particular models and their styles. And the philosophy behind the change was sound — people shouldn't need to understand the machinery to get good results from it.

Then, on launch day, the router failed. As chief executive Sam Altman explained the next day, the “autoswitcher” was out of action for part of the day, and the result was that GPT-5 “seemed way dumber.” Hard questions were being answered by the fast model instead of the one built for them. Users posted examples of basic mistakes. OpenAI fixed the router, said it would adjust how the routing decision was made, promised to be more transparent about which model was answering, and restored access to the previous model for paying subscribers who wanted it.

Nothing was wrong with the models. The problem sat entirely in the decision about which model to use — invisible when it works, and immediately visible when it doesn't.

That decision is called model routing. It is now one of the most important design choices in any product that uses AI, because it shapes cost, speed and quality at once. And you don't need to write code to understand it, question it, or make sure whoever builds your software has got it right.

Routing in one idea: triage

Think of a well-run hospital emergency department. A triage nurse sees every patient first and sends them to the right place. A sprained wrist goes to a nurse practitioner. Chest pain goes straight to a cardiologist. Nobody would send every patient to the cardiologist — it would be slow, expensive and wasteful — and nobody would send chest pain to the pharmacy counter.

Model routing is triage for AI requests. Some requests are simple: classify this email, extract a date, reformat a table, answer a question already answered many times before. Others are hard: reason through a contract clause, analyse a deviation report, write a careful answer to an upset customer. Small, fast models handle the first group well and cheaply. Large, capable models are worth their cost on the second. A router sits between your application and the models and makes that decision for every request.

The difference in price between the two ends is large. Across providers, the most capable models commonly cost many times more per token than their smaller siblings, and the reasoning models that “think” before answering add further cost through the hidden reasoning they generate. Send everything to the top model and you pay the cardiologist's rate for sprained wrists.

Four kinds of routing

In practice, routing decisions are made in four main ways, and good systems combine them.

1. By rule. The application knows what kind of task it is performing and sends it to a model chosen for that task: summaries to one, data extraction to another, final customer-facing answers to a third. Simple, predictable and easy to audit.
2. By difficulty. A small classifier looks at each request and estimates how hard it is, then picks a model accordingly. More flexible, and the kind of router GPT-5 relies on — but its judgement can be wrong, as launch day showed.
3. By data. Requests containing sensitive or regulated information go only to models hosted in approved locations, or to a model running inside your own environment. For organisations in the EU and the Gulf, this can matter more than cost.
4. For resilience. If one provider is slow, unavailable or over its limits, requests fail over to an equivalent model elsewhere, so the feature keeps working.

A fifth pattern, sometimes called a cascade, is worth knowing too. The request goes first to a small model; if that model's answer fails a quality check or it signals low confidence, the request is escalated to a stronger one. You pay the higher price only when the cheap answer isn't good enough.

What a routing plan looks like

Routing sounds abstract until you see it applied to a real feature. Take an assistant inside a customer portal for a contract manufacturer, answering sponsor questions about project status, documents and schedules. A routing plan is simply a table that says which kind of model handles which kind of work, and why.

Notice two rows. One uses no AI model at all, because ordinary software does the job better. Another sends the request to a human. Good routing is not about using AI everywhere; it is about putting each piece of work where it is handled best.

What routing saves

The savings can be substantial. Researchers behind RouteLLM, an open-source routing framework from the LMSYS group, reported in 2024 that routing easier queries to a cheaper model cut costs by more than 85% on one widely used benchmark while retaining about 95% of the strongest model's quality. Commercial tools now build routing in by default; Cursor, for example, offers an automatic mode that chooses a model for each request.

Your own savings will depend on your workload. A customer assistant where most questions are routine will benefit far more than a legal-analysis tool where almost every request is hard. That is why routing decisions should be based on testing with your real requests, not on a vendor's headline figure.

How AxON approaches this

Routing is built into AxON, our AI delivery platform, rather than bolted on. Each task in the features we build is assigned deliberately — routine work to smaller models, judgement-heavy work to stronger ones, sensitive data only to approved locations — and every answer records which model produced it. AxON is model-agnostic, so the pool of models can span providers and regions and change as better or cheaper options appear. The same discipline runs our own AI-assisted delivery, which is part of how we keep both build costs and our clients' running costs down.

How routing goes wrong

Routing is powerful precisely because it is invisible. That is also why its failures are easy to miss.

1. Silent misrouting. When the router sends hard questions to a weak model, nothing crashes. Answers simply get worse. Without monitoring, the first sign may be user complaints, exactly as OpenAI experienced.
2. Drift. Providers update their models. A route tested in January may behave differently in June. What was a safe choice for a task can quietly stop being one.
3. Inconsistency. Different models write differently. If customers receive answers from several models, tone, format and even substance can vary. For some uses this doesn't matter; for regulated communications it does.
4. Hidden overhead. A router is itself a piece of software. Classifying each request adds a little cost and a little delay, which must be smaller than the savings it produces.
5. Audit gaps. If nobody records which model produced which answer, you cannot investigate a bad one, reproduce it, or show a regulator how a decision was made.

Designing routing that holds

None of these problems is a reason to avoid routing. They are reasons to design it properly. When you commission software that uses AI, look for these seven things.

1. Routes tested on your own requests, with a quality bar agreed in advance, not borrowed from a benchmark.
2. A deliberate decision boundary that errs toward the stronger model when the stakes of a wrong answer are high.
3. A record of which model answered every request, and why.
4. Quality monitored per route, not only overall, so a weak route shows up quickly.
5. A manual override, so a route can be switched instantly if something goes wrong.
6. Failover across at least two providers for anything business-critical.
7. Transparency for users where it matters — OpenAI's own lesson.

A global pool, chosen without bias

Routing is also what makes it practical to use the whole market rather than a single supplier. The pool can include closed models from OpenAI, Anthropic, Google and xAI; European models from Mistral; open-weight families such as Meta's Llama, DeepSeek and Alibaba's Qwen; and regional models such as the UAE's Falcon and Jais, built for Arabic. A router lets you choose per task on quality, price and data location, and switch when the landscape changes — which it does every few months.

That independence is valuable in itself. A system hard-wired to one provider inherits that provider's prices, outages and policy changes. A system that routes across several can move.

When not to route

Routing is not always the right answer. Where an output must be exactly reproducible and fully validated, a fixed, tested model is usually preferable to a dynamic choice. In pharmaceutical manufacturing, Europe's draft GMP Annex 22 points in this direction for critical applications, expecting static, deterministic models and keeping generative AI to non-critical work under human oversight. The same logic applies anywhere a regulator or auditor needs to know exactly what produced a result.

Very small or very uniform workloads may not justify a router at all. If every request is similar in difficulty, one well-chosen model may be simpler and just as cheap.

Questions to ask whoever builds it

1. How do you decide which model handles each request, and how did you test that decision on our data?
2. What happens if the router sends a hard request to a weak model — how would we know?
3. Can we see which model produced any given answer?
4. Which providers and regions are in the pool, and can our sensitive data be restricted?
5. What happens when a provider is down or changes its model?

The bottom line

Model routing is triage for AI: send each request to the model that handles it well at the lowest reasonable cost. Done well, it cuts running costs sharply, improves resilience and frees you from dependence on one provider. Done carelessly, it degrades quality without anyone noticing — until the users do. You don't need to write code to tell the difference. You need to ask how routes are chosen, tested, recorded and monitored.

CTA: If you'd like to see how routing would apply to an AI feature you're planning, we'll map the tasks, recommend routes across providers, and show the cost and quality trade-offs in writing.

Frequently asked questions

What is model routing in AI?

It is the practice of sending each AI request to the most suitable model — usually a cheaper, faster model for simple tasks and a more capable one for complex tasks — based on rules, difficulty, data sensitivity or availability.

Does model routing reduce quality?

It shouldn't, if routes are tested on real requests and monitored. Poorly designed routing can reduce quality silently by sending hard requests to weak models.

Is routing only for large companies?

No. Many AI tools now include routing by default. Any business running AI features at meaningful volume can benefit, provided the routes are tested and monitored.