> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Route requests by model name

> Serve many providers and models from one route, and let clients pick one by naming it as provider/model.

The model router (`llm-model-router`) gives clients one base URL for every provider on a route, the way hosted LLM routers work. The client names a model as `<provider>/<model>`, for example `openai/gpt-5.5` or `anthropic/claude-sonnet-5-5`, and the gateway sends the request to a connection that serves that model. If requests to that connection fail, the gateway tries the next one. This topic shows you how to group a route's connections by provider, choose a strategy for each group, accept the gateway key from either SDK, and test the route. To compare the model router with the other routing plugins, see [Choose a routing approach](/configure/routing/choose-an-approach).

## Prerequisites

Before you begin, make sure you have the following:

* An LLM route with the connections you want to route across, one connection for each provider account or deployment. In the console, select **LLMs**, select the **Routes** tab, select **Create route**, and select every connection in **Connections**. Include the **GET /v1/models** endpoint. The console can't put connections with different API formats, such as OpenAI and Anthropic, on one route. To route across both, create the route with the admin API, as described in the **Admin API** tab of [Create the model router](#create-the-model-router).
* An allowed models list on every connection the router uses. When you connect the provider, clear **All current and future models** under **Models**, and select the models the connection serves. The router serves exactly the models on these lists.
* Permission to create plugin instances.
* No LLM Router or Adaptive Model Router on the route. A route can have only one routing plugin.
* The **llm-model-router** plugin in your plugin catalog. If it isn't there, ask a plugin administrator to make it available. If it's missing for everyone, contact support.

## Understand the configuration

The model router has two parts: provider groups, which this topic covers, and aliases, which [Define model aliases](/configure/routing/aliases) covers.

A **provider group** puts some of the route's connections under a name. The client uses that name as the prefix of the model, so with a group named `openai`, the model `openai/gpt-5.5` goes to the connections in that group. You choose the name. It doesn't have to match the provider's name, but it can't be empty or contain `/`.

Each group has a **strategy**:

| Strategy      | How the gateway orders the group's connections                                                                                                                                                |
| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `fallback`    | In the order you list them. The first connection gets every request, and the others are tried only when it fails.                                                                             |
| `loadbalance` | The first connection is picked at random in proportion to each connection's `weight`. If it fails, the rest are tried in the order you list them. Every connection needs a positive `weight`. |

A connection serves only the models on its **allowed models** list. When a client asks for `openai/gpt-5.5`, the gateway tries only the `openai` connections that list `gpt-5.5`, in the group's order. The router changes one thing in the request body: it sets `model` to the name the provider expects, `gpt-5.5` in this example. It doesn't translate between APIs, so a request goes only to connections whose provider serves the endpoint the client called.

The gateway moves to the next connection when the current one answers `429`, `500`, `502`, `503`, or `504`, or when the network connection to it fails or times out. It also moves on when a connection that signs its requests, such as an AWS connection, answers `401` or `403`. Any other answer, success or error, goes back to the client as the provider sent it. If the last connection answers one of the failover statuses, the client gets that answer as the provider sent it, with the `x-fabric-model` header. Before the gateway moves to the next connection, it retries the current one as that connection's retry settings allow. If the last connection can't be reached, the client gets `502`, or `504` if it timed out, with the message ``no candidate for model `<name>` is available``. If no connection is available to try, the client gets `503` with the same message. A streamed answer can fail over only until its first event reaches the client.

> **Note**
>
> An OpenAI Responses request that carries `previous_response_id` goes only to the first connection, with no failover, because the stored response lives at the provider account that created it. In a `loadbalance` group, that first connection is a weighted pick, so the request can reach an account that doesn't have the response and get a `404` from the provider. Clients that need failover there send the full input with `store: false`. The gateway applies this choice before it checks access, so if your key has no grant for that first connection, the request gets `403` even when a later connection would have allowed it.

## Create the model router

You can create the model router in the console or with the admin API.

#### Console

> **Note**
>
> The example below routes across OpenAI and Anthropic, so its route must be created with the admin API. On a route that you created in the console, every group uses the same API format.

The console has no form for this plugin, so you enter its configuration as JSON. To create the instance, do the following:

1. Select **Plugins**, find **llm-model-router**, and select **Use**.

2. Enter a **Name**, for example `model-router`.

3. Set **Scope kind** to `route`, and set **Scope name** to your route. The model router can be scoped only to a route.

4. In **Configuration**, select **Edit as JSON**, and then enter the configuration. The following example has an `openai` group that splits traffic three to one between two OpenAI connections, and an `anthropic` group that falls back from one Anthropic connection to another:

   ```json
   {
     "providers": {
       "openai": {
         "strategy": "loadbalance",
         "services": [
           { "name": "openai-us", "weight": 3 },
           { "name": "openai-eu", "weight": 1 }
         ]
       },
       "anthropic": {
         "strategy": "fallback",
         "services": [
           { "name": "anthropic-prod" },
           { "name": "anthropic-backup" }
         ]
       }
     },
     "aliases": {
       "claude-sonnet-5-5": [
         { "provider": "anthropic", "model": "claude-sonnet-5-5" },
         { "provider": "openai", "model": "gpt-5.5" }
       ]
     }
   }
   ```

   Each `name` is the name of a connection on the route. `aliases` is optional. For details, see [Define model aliases](/configure/routing/aliases).

5. Select **Save**.

If the configuration fails a check, Fabric doesn't save it and shows the reason. For the list, see [Fix a configuration that Fabric refuses](#fix-a-configuration-that-fabric-refuses).

#### Admin API

You can create the same configuration with the admin API. For the base URL and how to authenticate, see [Admin API](/configure/automation/admin-api). Replace `ADMIN_API_URL`, `TENANT`, and `ADMIN_API_KEY` in the following examples. `TENANT` is your organization ID, and a request with any other value gets `403` with `tenant mismatch`. The examples send the key in the `fg-tenant-api-key` header.

1. Give each connection an allowed models list in `llm.allowed_models`. A connection created through the admin API has this shape, with your usual upstream and credential fields:

   ```json
   {
     "name": "openai-us",
     "protocol": "llm",
     "llm": {
       "integration_id": "openai",
       "allowed_models": ["gpt-5.5"]
     }
   }
   ```

2. Create the route with its connections. The route body can carry the route's Key Auth instance in the `gateway_auth` field. Attach the model router separately. The path `/router/*` serves every endpoint under `/router`, so the route's base URL is your data plane URL followed by `/router`. A path without `*`, such as `/`, matches only that exact path, and every `/v1/...` request gets `404`:

   ```bash
   curl -X POST "ADMIN_API_URL/api/v1/TENANT/routes" \
     -H "fg-tenant-api-key: ADMIN_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "router",
       "protocol": "llm",
       "paths": ["/router/*"],
       "services": ["openai-us", "openai-eu", "anthropic-prod", "anthropic-backup"]
     }'
   ```

3. Create the model router as a plugin instance scoped to the route. The `config` takes the same JSON as in the console. This example leaves out `aliases`:

   ```bash
   curl -X POST "ADMIN_API_URL/api/v1/TENANT/plugin-instances" \
     -H "fg-tenant-api-key: ADMIN_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "model-router",
       "plugin_id": "llm-model-router",
       "scope": { "kind": "route", "name": "router" },
       "config": {
         "providers": {
           "openai": {
             "strategy": "loadbalance",
             "services": [
               { "name": "openai-us", "weight": 3 },
               { "name": "openai-eu", "weight": 1 }
             ]
           },
           "anthropic": {
             "strategy": "fallback",
             "services": [
               { "name": "anthropic-prod" },
               { "name": "anthropic-backup" }
             ]
           }
         }
       }
     }'
   ```

4. Create the route's Key Auth instance before you send traffic, unless you added it in `gateway_auth` in step 2. Without it, the route doesn't require a gateway key. The `headers` list accepts the key from either SDK:

   ```bash
   curl -X POST "ADMIN_API_URL/api/v1/TENANT/plugin-instances" \
     -H "fg-tenant-api-key: ADMIN_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "router-key-auth",
       "plugin_id": "key-auth",
       "scope": { "kind": "route", "name": "router" },
       "config": {
         "header": "fg-api-key",
         "headers": ["Authorization", "X-API-Key"]
       }
     }'
   ```

## Accept the gateway key from either SDK

SDKs send their API key in their own header: the OpenAI SDK sends `Authorization: Bearer <key>`, and the Anthropic SDK and Claude Code send `x-api-key`. To let clients put their gateway key where their SDK already puts it, add a `headers` list to the route's Key Auth instance:

1. Select **Plugins**, select the **Configured** tab, and then open the Key Auth instance that's scoped to your route.

2. Select **Edit as JSON**.

3. Keep the existing fields, and add the `headers` list:

   ```json
   {
     "header": "fg-api-key",
     "headers": ["Authorization", "X-API-Key"]
   }
   ```

4. Select **Save**.

With this list, Key Auth reads the key from the headers on it instead of from `fg-api-key`, so a caller that sends only `fg-api-key` gets `401`. The list can name any headers, so to accept all three, add `fg-api-key` to it. A request that carries two different values in these headers gets `401` with `conflicting credentials`.

On a model router route, the gateway removes every provider credential header the client sent (`Authorization`, `x-api-key`, and `x-goog-api-key`) before it calls the provider, so a client that sends its own provider key instead of a gateway key is refused, not passed through. It also removes the `x-gateway-service` and `x-amr-session-id` headers and every header that starts with `x-fabric-`.

The gateway picks a request's namespace from the `fg-api-key` header, and serves a request without that header from the **Default** namespace. A key that the gateway doesn't know gets `401`, never the **Default** namespace. So a route that callers reach with their SDK's own header must be in the **Default** namespace. For namespaces, see [Manage namespaces](/configure/organization/namespaces).

## Fix a configuration that Fabric refuses

Fabric checks the configuration when you save the model router, its route, or one of the route's connections, and refuses a save that breaks any of the following rules:

| Rule                                                                                                                                                                                                          | What to fix                                                                        |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| The instance is scoped to an LLM route.                                                                                                                                                                       | Set **Scope kind** to `route` and pick an LLM route.                               |
| `providers` has at least one group, each group has at least one connection, and each group name is non-empty with no `/`.                                                                                     | Add a connection to the group, or rename or remove it.                             |
| Every connection in a group is on the route, is an LLM connection set up for a provider, and is in only one group.                                                                                            | Add the connection to the route, or remove it from the other group.                |
| Every connection in a group has an allowed models list, with no empty or repeated entries.                                                                                                                    | Edit the connection's allowed models.                                              |
| `strategy` is `fallback` or `loadbalance`. Every connection in a `loadbalance` group has a positive `weight`, the weights add up to a number that fits in an integer, and a `fallback` group has no `weight`. | Fix the strategy or the weights.                                                   |
| The route has no LLM Router or Adaptive Model Router.                                                                                                                                                         | Remove the other routing plugin.                                                   |
| No two connections on the route serve the same path in different API formats.                                                                                                                                 | Move one of the connections to another route.                                      |
| The instance is at route scope, and its failure mode is the default, which blocks a request the router didn't handle.                                                                                         | If you set `on_error` through the admin API, remove it or set it to `fail_closed`. |
| Every alias name is non-empty, has no `/`, and isn't the name of a provider group.                                                                                                                            | Rename the alias.                                                                  |
| Every alias has at least one target and no target twice, and each target names a provider group and a model that a connection in that group allows.                                                           | Fix the target, or allow the model on a connection.                                |

A connection on the route can be in no group. The router never picks it, and it doesn't appear in the models list. To add a connection, first add it to the route, and then add it to a group in the router's configuration. To remove one, take it out of its group first, and then out of the route.

## Test the route

Replace `BASE_URL` with your data plane URL followed by the route's base path, for example `https://gateway.example.com/router`. To copy it, see [Get the base URL](/use/authenticate#get-the-base-url). Point the OpenAI SDK at `BASE_URL/v1`, and the Anthropic SDK at `BASE_URL`.

To list the models the route serves, send a request to `/v1/models` with your gateway key. The examples send the key in `Authorization`. If you didn't add the `headers` list to Key Auth, send it in `fg-api-key` instead.

```bash
curl "BASE_URL/v1/models" \
  -H "Authorization: Bearer GATEWAY_KEY"
```

The gateway answers this itself, without calling a provider. The list has one entry for each `<provider>/<model>` and each alias that the caller can use, for example `openai/gpt-5.5` and `claude-sonnet-5-5`.

For the example configuration, the response looks like this:

```json
{
  "object": "list",
  "data": [
    { "id": "anthropic/claude-sonnet-5-5", "object": "model", "created": 0, "owned_by": "anthropic" },
    { "id": "claude-sonnet-5-5", "object": "model", "created": 0, "owned_by": "fabric" },
    { "id": "openai/gpt-5.5", "object": "model", "created": 0, "owned_by": "openai" }
  ]
}
```

An alias has `owned_by` set to `fabric`. If you can use none of the models, `data` is empty.

Then send a chat completion request that names one of the models:

```bash
curl -i "BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.5",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'
```

A successful request returns `200 OK` with a chat completion from the provider. The response headers include the following:

```text
HTTP/1.1 200 OK
x-fabric-model: openai/gpt-5.5
```

The `x-fabric-model` header names the `<provider>/<model>` that answered. For the errors a client can get, see [Choose a model](/use/llm/choose-a-model).

## Serve every namespace from a Default route

A model router route in the **Default** namespace can also serve the other namespaces in your organization, so every team uses one set of provider connections. This happens when the route meets all of the following conditions:

* It's an LLM route in the **Default** namespace.
* It has an enabled model router instance scoped to the route, with no match conditions.

A request from another namespace reaches this route when that namespace has no route of its own at the request's path. The namespace's own route always wins. The request also must not match a more specific **Default** route that has no model router: if it does, the request gets `404`. The request works as follows:

* **Authentication.** The caller sends a gateway key from its own namespace in the `fg-api-key` header. The namespace must authenticate it with a namespace-wide plugin, such as a Key Auth instance with **Scope kind** set to `global`. A new namespace doesn't have one, so create it in that namespace. Without one, the request gets `401`. The **Default** route's own Key Auth checks only **Default** callers.
* **Connections.** The caller reaches only the route's **Open** connections, never its **Gated** ones. If none of the connections for the model is open, the request gets `403`.
* **Plugins.** The **Default** route's plugins and its connections' plugins run, except authentication plugins. The namespace's plugins at global, consumer, and consumer group scope run too, except access control.
* **Usage and limits.** Usage is recorded under the namespace that sent the request, with the **Default** route and connection names. Limits and budgets on the **Default** route and connections are one pool shared by every namespace that uses the route. Set per-namespace limits in the namespace.

To stop serving other namespaces, remove the model router instance from the route. Requests from other namespaces then get `404`.

## Next steps

* [Define model aliases](/configure/routing/aliases)
* [Choose a model](/use/llm/choose-a-model)
* [Use the OpenAI-compatible API](/use/llm/openai-compatible)