# What is LLM routing?

> How LLM routing selects models by request requirements, cost, quality, latency, context, and policy.

Source: https://ailoot.ai/blog/en/what-is-llm-routing/
Canonical: https://ailoot.ai/blog/en/what-is-llm-routing/
Language: en
Publisher: Ailoot (https://ailoot.ai/)
Translation: guide-100040
Revision: sha256-c4bd9c59d03de6baeb2ba472bac43e9e928579a698a78a1ef0f606a13696751a


**LLM routing** is the practice of sending each AI request to the model and provider best suited to it, instead of hardcoding every call to a single model. A router evaluates the request against your goals, cost, quality, latency, context length, and policy, and dispatches it to the option that best fits.

Most teams start by wiring their app directly to one provider, usually OpenAI. That works until it doesn't: a cheaper model would handle 60% of your traffic just as well, a provider has an outage, a request needs a longer context window, or a regulated workload can't leave a jurisdiction. A router solves all of these behind one endpoint.

## How LLM routing works

You point your existing OpenAI-compatible SDK at the router instead of the provider. For each request, the router:

1.  **Builds a candidate set** of models that can serve the request (right capabilities, context window, and availability).
2.  **Scores candidates**against your objective, for example "cheapest option above a quality floor" or "lowest measured latency."
3.  **Applies constraints** like budgets, allowed providers, data residency, and per-key model allow-lists.
4.  **Dispatches and falls back**: if the first choice errors or times out, it retries the next best candidate automatically.

## Why teams route instead of hardcoding

- **Cost.** Route easy requests to cheap models and reserve premium models for the ones that need them.
- <strong>Reliability.</strong>Automatic failover across providers means one vendor's outage doesn't take your product down.
- **Flexibility.** Swap or add models without shipping code, because routing lives in a policy, not your app.
- **Governance.** Enforce budgets, residency, and quality floors centrally instead of per service.

## Routing vs. a simple fallback

A fallback ("if OpenAI fails, try Anthropic") is the most basic form of routing, but real routing optimizes on **every** request, not just failures, choosing the best option up front by cost and quality, not only when something breaks.

## Put this guide to work with Ailoot

Connect AI models to your applications, agents and automations through one Ailoot API. Choose the model that fits your task and follow the [Ailoot documentation](https://ailoot.ai/docs/) to get started.


## Ailoot · The company and its project portfolio

Ailoot combines established operating experience, a large client base, stable service and ongoing KPI control.

The company and its project portfolio: millions of customers; monthly sales in the billions.

Ailoot brings that operating experience to API access for AI features in customer products, agents, applications and automations.
