Skip to navigation

Route LLM requests across models and providers

Routing plugins decide which model or provider serves each request on a route. The Routing category of the plugin catalog includes three plugins. The model router (llm-model-router) sends each request to a connection that serves the model the client names. The Adaptive Model Router picks a model and reasoning effort for each request. The LLM Router controls which connections handle requests and what happens when one fails. This topic compares the plugins and explains how to check which model answered a request.

Understand the plugins

Each routing plugin attaches to a route. To find them, select Plugins, filter by the Routing category, and select Use next to the plugin.

PluginWhat it does
Adaptive Model RouterSelects an LLM model and reasoning effort for each request. It uses a decision service, which rates the request and returns probabilities that the plugin uses to choose.
llm-model-routerSends each request to a connection that serves the model the client names, as <provider>/<model> or an alias, and fails over within the provider group.
LLM RouterChooses which of the route’s connections handles each request. Its modes either choose among the connections you list or, in lightyear mode, send requests through Lightyear.

The llm-model-router plugin gives clients one base URL for every provider on a route. The client names a model as <provider>/<model>, for example openai/gpt-5.5, or by an alias such as claude-sonnet-5-5. The gateway sends the request to a connection that serves that model, and fails over to the next one in the provider group. Clients can list the models they can use with GET /v1/models. The console has no form for this plugin, so you configure it as JSON. For the steps, see Route requests by model name. To decide which routing plugin fits a route, see Choose a routing approach.

To set up a plugin, see the following topics:

Check which model answered

After you save a routing plugin, send test requests to the route. To confirm which model answered, check the model field in the response. In Traces, a request that uses adaptive model routing appears with its own request to the decision service, followed by the request to the provider. For the steps, see Inspect traces.

Next steps