Skip to navigation

Choose a routing approach

Fabric AI Gateway has three plugins that decide where an LLM request goes. They answer different questions: which provider serves the model the client named, what happens when a provider fails, and which model a request needs. This topic compares them so you can pick one for each route.

Compare the plugins

Model router (llm-model-router)LLM RouterAdaptive Model Router
Who picks the modelThe client, by name, for example openai/gpt-5.5 or an aliasThe clientThe plugin, based on how difficult the request is
Providers on one routeSeveral. The client reaches any of them through one base URL.Several, all serving the models the client asks forOne provider per request. It chooses among that provider’s models.
FailoverPer provider group, in order (fallback) or by weight (loadbalance). An alias can also fall back to another provider.Across the connections you list, in priority orderNo failover
Model list for clientsGET /v1/models lists every model and alias the caller can useNoNo
Gateway keyfg-api-key, or the header each SDK already sends (Authorization: Bearer or x-api-key)fg-api-keyfg-api-key
Set up in the consoleEdit as JSON on the plugin instanceA formA form
Use it whenYou want one base URL for many providers and models, the way hosted LLM routers workYou want a route to keep answering during a provider outageYou want to cut cost by sending simple requests to smaller models

Model router

The model router lets the client choose both the provider and the model in the model field. You group the route’s connections by provider, and each connection serves the models you allowed on it. Clients can list what’s available with GET /v1/models, and an alias lets them use a plain name such as claude-sonnet-5-5 that you can point at different providers later. For the steps, see Route requests by model name.

The model router doesn’t translate between APIs. A model answers only on the endpoints its provider serves, so an Anthropic model works on /v1/messages, and on /v1/chat/completions only if that provider also offers an OpenAI-compatible endpoint.

LLM Router

The LLM Router keeps the client’s model name as it is and decides which of the route’s connections handles the request. Use it to fail over to a second provider that serves the same models. For the steps, see Fail over across providers with the LLM Router. The LLM Router also has a Lightyear mode, which sends requests through the Lightyear service. For details, see Route requests through Lightyear.

Adaptive Model Router

The Adaptive Model Router replaces the model the client asked for with one that fits the request. It asks a decision service to rate each prompt, and then picks a smaller or larger model from the same provider. For the steps, see Route requests with the Adaptive Model Router.

Use one routing plugin per route

Each of these plugins builds the list of connections that a request tries, so a route can use only one of them. Fabric refuses to save a model router on a route that already has an LLM Router or an Adaptive Model Router, and the reverse.

The model router covers most of what the LLM Router does: each provider group has its own fallback or loadbalance strategy, and an alias can fall back across providers. If you need both a choice of model and failover, use the model router.

Fabric doesn’t refuse an LLM Router and an Adaptive Model Router on the same route, but requests on that route then fail with 503. Use only one of them on a route.

Next steps