> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Route requests with the Adaptive Model Router

> Rate each LLM request with a decision service and send it to the model that fits its difficulty.

The Adaptive Model Router picks a model and reasoning effort for each LLM request on a route. This topic shows you how to connect a decision service, create the plugin instance, and test it. To compare it with the other routing plugins, see [Route LLM requests across models and providers](/configure/routing).

## Prerequisites

Before you begin, make sure you have the following:

* A route that accepts authenticated requests. For the steps, see [Connect an LLM provider](/configure/connections/llm-providers).
* No other routing plugin on the route. If a route has two, such as the LLM Router and the Adaptive Model Router, requests fail with `503`.
* A provider connection with no model allowlist, for the plugin to route to. Fabric refuses an Adaptive Model Router whose **Connection** has a model allowlist, and refuses adding an allowlist to a connection that an Adaptive Model Router uses. The decision connection can have one.
* Permission to create plugin instances.
* A decision service. This topic explains how to set one up.

## Add the Adaptive Model Router

Models differ in how much they reason before they answer. Smaller models answer quickly and cost less. Larger models reason more, which uses more tokens and costs more, even for a request that doesn't need that effort. With adaptive model routing, the gateway rates each prompt before it reaches the provider and sends the prompt to the model that fits its difficulty. A simple greeting goes to a small model even if the client asked for a large one.

The plugin does this with a *decision service*: a model that returns probabilities instead of text. For example, a decision service can read a support ticket and report how likely it is to be about billing, technical problems, or something else. The Adaptive Model Router uses the same approach to rate how difficult a prompt is.

To set up adaptive model routing, connect a decision service, and then create the plugin instance.

### Connect a decision service

You bring your own decision service. It must be a Jev endpoint. The gateway sends a request to the endpoint's `/v1/systemone` path, so you can connect Jev, or a Jev endpoint that you host yourself. Other decision APIs aren't supported.

If your security requirements don't allow traffic to leave your infrastructure, host both the decision model and the LLM yourself. Then the gateway routes everything through your own infrastructure, and no data goes to another provider. To connect an endpoint that you host, specify your own target URL when you create the connection. The target URL must be an HTTPS origin with no path.

To connect a hosted decision service such as Jev, do the following:

1. In the sidebar, select **LLMs**, select the **Providers** tab, search for the provider, and click **Connect**. For details about the **Create LLM connection** page, see [Connect an LLM provider](/configure/connections/llm-providers).
2. In **Upstream auth**, enter the provider's API key.
3. In **Models**, allow the models that the decision service offers.
4. In **Gateway route**, keep **Create gateway route** selected to expose the connection on a route. You can reuse a gateway key that you already created.
5. Click **Create connection**.

Fabric stores the decision service's API key, so you don't enter it on the plugin or on the route.

### Create the plugin instance

To create the plugin instance, do the following:

1. Select **Plugins**, find **Adaptive Model Router** in the **Routing** category, and click **Use**.

2. Enter the following values:

   | Field                   | Value             | Description                                                                                                                                                                                           |
   | ----------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | **Name**                | `adaptive-router` | Must be unique within your organization. You can't change it later.                                                                                                                                   |
   | **Scope kind**          | `route`           | The Adaptive Model Router can be scoped only to a route.                                                                                                                                              |
   | **Scope name**          | `anthropic-route` | The route whose requests the plugin routes.                                                                                                                                                           |
   | **Connection**          | `anthropic`       | The connection that receives the routed request. The list shows the connections that the route exposes. If the route has several connections, choose the one to use. It can't have a model allowlist. |
   | **Decision connection** | `jev`             | The connection that rates each prompt. Fabric resolves it across your organization, so you don't need to attach it to this route.                                                                     |

3. Click **Advanced settings**, and then set the following options:

   | Field                         | Value        | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
   | ----------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | **Jev model**                 | `jev-1.13.0` | The model identifier that the gateway sends to the decision service. Pin a version so routing doesn't change under your tuned thresholds when a newer version is released.                                                                                                                                                                                                                                                                                                                                                                                             |
   | **Max input characters**      | `32000`      | How much of the request the gateway sends to the decision service. The default is 32000.                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
   | **Default max output tokens** | `4096`       | The output token limit that the gateway applies to the routed request to the chosen model, not to the decision service's call. The gateway applies the smaller of this value and the client's own limit, and a small cap can also lower the reasoning effort. The default is 4096.                                                                                                                                                                                                                                                                                     |
   | **Session stickiness**        | Off          | When on, a conversation stays on the model chosen in its first turn. This keeps the provider's prompt cache warm and skips a decision call on later turns. It applies within one session, such as a back-and-forth with a coding agent, and it needs the client to send a session ID header, such as `X-Amr-Session-ID`, `X-Claude-Code-Session-ID`, `Thread-ID`, `Session-ID`, or `X-Session-ID`. Requests without one aren't pinned. A pin expires 24 hours after its last use. If the data plane has no distributed store, requests with a session ID return `503`. |
   | **Include system messages**   | Off          | When on, the gateway also sends system messages to the decision service. Many clients add long system instructions to every request, which a decision service usually doesn't need. Turn it on if the instructions affect how a prompt should be rated.                                                                                                                                                                                                                                                                                                                |
   | **Include tool descriptions** | Off          | When on, the gateway also sends tool definitions to the decision service. Clients that have several MCP servers attached send many tool definitions with each request. Turn it on only if tool details affect how a prompt should be rated.                                                                                                                                                                                                                                                                                                                            |

4. Click **Save**.

> **Note**
>
> The **Include system messages** and **Include tool descriptions** options send more of your request content to the decision service. By default, the gateway sends the conversation text up to **Max input characters**, without system messages or tool descriptions, and waits up to 5 seconds for an answer. Leave them off unless your routing needs that content.

### Test the plugin

To test the routing, send the same request that you used to test the route, and ask for a larger model. For a simple prompt such as `Hello, world!`, the response comes back from a smaller model than the one in your request. The gateway sent the prompt to the decision service, which rated it as simple, and the plugin changed the model before the request reached the provider.

To confirm which model answered, see [Check which model answered](/configure/routing#check-which-model-answered).

## Next steps

* Keep requests working during a provider outage. For the steps, see [Fail over across providers with the LLM Router](/configure/routing/failover).
* Compare cost and token usage across models on the **LLM** tab of [Review analytics](/use/monitor/analytics).