> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# LLM Router plugin

> Reference for the LLM Router plugin, which sends a route's LLM requests to one connection, a fallback chain, or a weighted mix.

The LLM Router plugin decides which of a route's connections handle each LLM request and what happens when one fails. It can send every request to one target, walk an ordered fallback chain, pick a target by weight and spill over to the rest, or, in `lightyear` mode, send requests through Lightyear. It only routes across connections that are already on the route.

A client can send the `x-gateway-service` header to pick one connection on the route for a request. That request goes to that connection only and doesn't fail over. A header that names a connection that isn't on the route gets `400`. The router ignores the header in `lightyear` mode.

## Where it runs

|             |                                                         |
| ----------- | ------------------------------------------------------- |
| Protocols   | LLM routes. On other protocols the plugin does nothing. |
| Scope kinds | `route` only                                            |
| Phase       | Request                                                 |

## Configuration

**`mode`** `string` — default: single

`single` sends requests to exactly one target. `fallback` tries targets by `order`, lowest first, and moves on when one fails. `loadbalance` picks the first target at random in proportion to `weight`, and then tries the others in list order. `lightyear` sends requests through Lightyear and takes no targets. For more information, see [Route requests through Lightyear](/configure/routing/lightyear).

---

**`credentials`** `string`

Valid only in `lightyear` mode. Set it to `client` to forward the caller's own provider credential instead of the one saved on the connection. The console shows this as **Use the client's provider credentials**.

---

**`targets`** `array of objects`

The connections to route to. The control plane refuses an empty list on save, except in `lightyear` mode, which takes no targets.

---

**`targets[].service`** `string` — required

A connection on this route. Each connection can appear once.

---

**`targets[].order`** `integer` — default: List order

The position in a `fallback` chain. Equal values keep list order.

---

**`targets[].weight`** `integer`

The share of first attempts in `loadbalance` mode. Must not be negative, and must be positive in `loadbalance` mode. The control plane doesn't currently catch an omitted weight, so such a route returns `500`.

---

**`retry.max_attempts`** `integer` — default: No retries

Attempts on one target before moving to the next, counting the first, from 2 to 10.

---

**`retry.initial_backoff_ms`** `integer` — default: Gateway default

The first retry delay in milliseconds: `0`, or 50 to 10,000.

---

**`on_status_codes`** `array of integers`

Extra status codes, from 400 to 599, that trigger a retry or fallback. `429` and `503` always do. A `4xx` code other than `429` triggers a fallback but isn't retried on the same target.

---

**`on_status_match`** `object`

A range-aware alternative to `on_status_codes`. You can't set both. It has an `operator` of `in` or `not_in`, and a `codes` array of single codes or inclusive ranges between 400 and 599, such as `[429, "500-599"]`. With `in`, the router falls back on the listed codes and on the codes that always trigger fallback. With `not_in`, it falls back on every code from 400 to 599 except the listed ones, and you can't exclude `429` or `503`. You also can't exclude `500`, `502`, or `504` while `allow_nonidempotent_failover` is on. The console shows this as **Fall back on** and **Status codes**.

---

**`allow_nonidempotent_failover`** `boolean` — default: true

Also fails over on `500`, `502`, and `504`. A fallback can repeat a request that the first provider already processed.

---

**`enrolled_consumers`** `array of strings` — default: Every caller

Applies the targets only to these consumers. Other callers get the gateway's default routing.

---

A configuration that fails these checks makes every request on the route return `500`. The control plane rejects such a configuration when you save it. A deployment that has no control plane doesn't check it on save.

## Example

```json
{
  "name": "llm-router-fallback",
  "plugin_id": "llm-router",
  "scope": { "kind": "route", "name": "openai-route" },
  "config": {
    "mode": "fallback",
    "targets": [
      { "service": "openai", "order": 1 },
      { "service": "azure-openai", "order": 2 }
    ],
    "on_status_codes": [408]
  }
}
```

## Related guides

* [Fail over across providers with the LLM Router](/configure/routing/failover)
* [Route LLM requests across models and providers](/configure/routing)