> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Limits and defaults

> Body size caps, timeouts, retry and health-check defaults, and streaming limits in Fabric AI Gateway.

Fabric AI Gateway applies fixed size caps to the requests and responses it reads, and default timeouts and retry settings to the upstreams it calls. This page lists those values. Limits that you configure yourself, such as rate limits, token budgets, and budgets, are on their own pages: [Rate limits](/configure/access-and-limits/rate-limits) and [Budgets](/configure/access-and-limits/budgets).

## Request and response size

| What                                                           | Limit                     | What happens above it                                                                                                                                                                                         |
| -------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Request headers                                                | 1 MiB                     | The request is refused with `431` before the gateway handles it.                                                                                                                                              |
| MCP request body                                               | 8 MiB                     | Refused with `413` `mcp_body_too_large`.                                                                                                                                                                      |
| LLM request body that the gateway reads                        | 10 MiB                    | The gateway doesn't read the body, so it doesn't know the model. If the connection allows only some models, the request is refused with `400` `Could not determine a model`. Otherwise it's forwarded unread. |
| LLM request body on a route with the `llm-model-router` plugin | 10 MiB (10,485,760 bytes) | Refused with `413` `request body too large`. A body of exactly 10 MiB is accepted.                                                                                                                            |
| Request body kept for retries and failover                     | 10 MiB                    | Larger bodies are still sent to the upstream, but the gateway doesn't retry them or fail over.                                                                                                                |
| REST request body that a plugin reads                          | 1 MiB                     | The plugin runs without the body. A plugin that needs the body and is set to fail closed fails, and the client gets `502` `policy plugin failure`.                                                            |
| REST response held for transformation or validation            | 1 MiB                     | The response is passed on unchanged, and the response transformation and validation are skipped.                                                                                                              |
| LLM response read for token counting                           | 10 MiB                    | The client still gets the whole response. Usage beyond the cap isn't read.                                                                                                                                    |
| First event of an LLM stream                                   | 1 MiB                     | The gateway holds the first event until it's complete, so that it can still fail over if the event is an error.                                                                                               |

The client always receives the full response body. The caps only limit what the gateway keeps in memory.

## Timeouts

Each connection can set its own timeouts. When it doesn't, the gateway uses these defaults:

| Timeout                                | Default    | Notes                                                                                                                                                                                                         |
| -------------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Connect to the upstream                | 15 seconds | A connection's connect timeout can only shorten it.                                                                                                                                                           |
| TLS handshake with the upstream        | 10 seconds | Replaced by the connection's connect timeout when one is set.                                                                                                                                                 |
| Wait for response headers              | None       | Set a read timeout on the connection to limit it. When it fires, the client gets `504`.                                                                                                                       |
| Gap between reads of the response body | None       | The read timeout also applies here. An upstream that keeps sending, such as a live event stream, is never cut off by it.                                                                                      |
| Upload from the client                 | None       | Set a write timeout on the connection to limit how long an upload may stall. When it fires, the client gets `408`.                                                                                            |
| Whole attempt                          | None       | The connection's overall timeout covers one attempt, including its retries and the whole response body. When it fires before a response starts, the client gets `502` on REST routes and `504` on LLM routes. |

The connect, read, and write timeouts each accept 0 to 600,000 milliseconds. A value of 0 means not set.

## Retries and failover

| Setting                                       | Default                                                                                                             |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| Retries against the same upstream             | None. A retry policy on the connection turns them on.                                                               |
| What a retry policy retries                   | Network errors, including timeouts and resets. A policy can also retry `5xx` answers or resets only.                |
| Wait between retries                          | Starts at 100 ms and doubles each time, up to 10 seconds.                                                           |
| Requests with signed credentials, such as AWS | Always get a second try with fresh credentials after a `401` or `403`.                                              |
| Status codes that trigger LLM Router failover | `429` and `503`, plus `500`, `502`, and `504` when nonidempotent failover is on. For LLM routes it's on by default. |
| Retry after the response starts               | Never. Once the gateway sends response headers, it doesn't retry or fail over.                                      |

For how to configure failover, see [Fail over across providers with the LLM Router](/configure/routing/failover).

## Upstream health

Health checks and the circuit breaker run only on connections that configure them. Without them, the gateway doesn't take a failing upstream out of rotation. When a connection configures them without setting these values, the gateway uses these defaults:

| Setting                                                  | Default                                                   |
| -------------------------------------------------------- | --------------------------------------------------------- |
| Failures in a row before an upstream is marked unhealthy | 3                                                         |
| Successes in a row before it's healthy again             | 2                                                         |
| Active health check interval, when a check path is set   | 10 seconds                                                |
| Active health check timeout                              | 5 seconds                                                 |
| Failures in a row before the circuit breaker opens       | 5                                                         |
| Time the circuit stays open                              | 30 seconds. After that, one trial request is let through. |

## Connections to upstreams

| Setting                                 | Value      |
| --------------------------------------- | ---------- |
| Idle connections closed after           | 90 seconds |
| Idle connections kept per upstream host | 10         |
| Minimum TLS version to upstreams        | TLS 1.2    |

## Streaming

| What                                         | Value                                                                                                                                                                                                                                  |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Server-sent events and AWS event streams     | Passed on event by event, as soon as each one is complete.                                                                                                                                                                             |
| Newline-delimited JSON streams on LLM routes | Passed on as the bytes arrive, in chunks of up to 32 KiB.                                                                                                                                                                              |
| MCP notification streams per caller          | 10 on each data plane instance. An 11th gets a stream with a 30-second retry hint, and is then closed. On revision 2026-07-28 (`subscriptions/listen`), the 11th request gets HTTP `503` with JSON-RPC `-32603` and `Retry-After: 30`. |
| Keep-alive on MCP notification streams       | A comment every 30 seconds.                                                                                                                                                                                                            |

## MCP tool lists

| What                                                             | Value                                                             |
| ---------------------------------------------------------------- | ----------------------------------------------------------------- |
| Timeout for each MCP server when the gateway builds `tools/list` | 30 seconds. A server that doesn't answer is left out of the list. |
| Tool description length                                          | Cut to 4,096 bytes.                                               |

## Other limits

| What                                                    | Value                                                         |
| ------------------------------------------------------- | ------------------------------------------------------------- |
| `X-Request-Id` that the gateway accepts from the client | 256 characters. A longer one is replaced with a generated ID. |
| Adaptive Model Router session ID                        | 256 characters. Pins last 24 hours from the last use.         |

## Next steps

* Look up what each error status means in [Gateway error codes](/reference/error-codes).
* Set request and token limits for your callers in [Rate limits](/configure/access-and-limits/rate-limits).
* Size a data plane that you run yourself in [Scale and high availability](/run/scale-and-high-availability).