> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Fail over across providers with the LLM Router

> List connections in priority order so that a request that fails on one provider moves to the next.

The LLM Router controls which connections handle requests on a route and what happens when one fails. This topic shows you how to configure fallback across providers. To compare it with the other routing plugins, see [Route LLM requests across models and providers](/configure/routing).

## Prerequisites

Before you begin, make sure you have the following:

* A route that accepts authenticated requests. For the steps, see [Connect an LLM provider](/configure/connections/llm-providers).
* Permission to create plugin instances.
* At least two connections on the route.
* No model router or Adaptive Model Router on the route. A route can have only one routing plugin.

## Configure fallback

Fallback routing keeps requests from failing when a provider has an outage. You list connections in priority order. The gateway tries the first one. If it fails, the gateway moves down the list until one of them answers.

For example, a route can list OpenAI first, and then the same OpenAI models hosted on Azure or on Amazon Bedrock Mantle. If a request to OpenAI fails, the gateway sends it to the next connection, which serves the same models. Fabric sends every target the same request, including the `model` name, so each target must accept that name. To send each provider its own model ID, use [model aliases](/configure/routing/aliases) instead.

To create an LLM Router instance for fallback, do the following:

1. Select **Plugins**, find **LLM Router** in the **Routing** category, and select **Use**.

2. Enter a **Name**, for example `llm-router-fallback`. The LLM Router can be scoped only to a route, so set **Scope kind** to `route` and **Scope name** to the route you want to configure.

3. For **Mode**, select `fallback`.

4. Configure the following settings:

   | Field                            | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
   | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | **Targets**                      | The connections to try, in priority order. The top target has the highest priority. Select the arrows to reorder targets, **Add target** to add one, and the trash icon to remove one.                                                                                                                                                                                                                                                                                                                                                                                      |
   | **Attempts per target**          | How many times Fabric tries a provider before it moves to the next target. The default is **No retries (1 attempt)**: Fabric moves to the next target unless the service has its own retry settings. If you allow retries, set between 2 and 10 attempts, counting the first request. A `4xx` status other than `429` moves to the next target without a retry. Each extra attempt adds latency, so use retries sparingly.                                                                                                                                                  |
   | **Initial retry delay (ms)**     | Appears when you allow retries. How long Fabric waits before the first retry on the same target, from 50 to 10,000 ms. The default is 100. The delay doubles after each retry.                                                                                                                                                                                                                                                                                                                                                                                              |
   | **Fall back on**                 | Which responses make Fabric retry on the current target and then fall back. **Default status codes** uses `429` and `503`, and also `500`, `502`, and `504` when nonidempotent failover is on. You can also select **Any 4xx or 5xx**, **Any 4xx**, **Any 5xx**, **Rate limits (429) and any 5xx**, **These status codes…**, or **All status codes except…**.                                                                                                                                                                                                               |
   | **Status codes**                 | Appears when you select **These status codes…** or **All status codes except…**. Enter codes or ranges between 400 and 599, separated by commas, for example `404, 500-599`. With **All status codes except…**, Fabric falls back on every code from 400 to 599 that you don't list, for example `400-401, 404, 422`.                                                                                                                                                                                                                                                       |
   | **Allow nonidempotent failover** | Controls whether Fabric also falls back on `500`, `502`, and `504` responses, and on timeouts after the request was sent, when the first provider may already have processed the request. It applies to every request on the route. Choose **Default (LLM: fail over)**, **Always fail over**, or **Never**. For LLM routes, the default is to fail over, which adds `500`, `502`, and `504` to the status codes that trigger fallback. A fallback can repeat a request that the first provider already processed, so select **Never** if duplicate requests are a problem. |

5. Select **Save**.

`429` and `503` always trigger fallback, so you can't exclude them. You can't exclude `500`, `502`, or `504` while nonidempotent failover is on. Set it to **Never** first.

Fabric retries a network error, a `429`, or a `5xx` response that triggers fallback on the same target, up to **Attempts per target**. It doesn't retry any other `4xx` response on the same target, because the same target would give the same answer. It falls back to the next target instead.

For streaming responses, Fabric retries a retryable first event before it sends any bytes. After the first valid event reaches the client, Fabric doesn't retry or fall back.

A client that sends the `x-gateway-service` header reaches only the service it names, with no fallback. If the header names a service that isn't on the route, the client gets `400`.

If every target fails, the client gets the last target's answer as the provider sent it. If the last target can't be reached, the client gets `502`, or `504` if it timed out. If no target is available to try, the client gets `503`.

To confirm which model answered, see [Check which model answered](/configure/routing#check-which-model-answered). The response doesn't name the service that answered. To see which target served a request, open the request in **Traces**. For details, see [Inspect traces](/use/monitor/traces).

## Next steps

* Pick a model for each request based on its difficulty. For the steps, see [Route requests with the Adaptive Model Router](/configure/routing/adaptive-routing).