Skip to navigation

Handle errors and retries

A failed request through Fabric AI Gateway can fail in the gateway or in the upstream behind it. When the upstream answers with an error, the gateway passes it to you unchanged. When the gateway itself stops the request, the response shape depends on the route’s protocol and on which step stopped it. This topic explains how errors look on each protocol, which ones are safe to retry, and how rate limits and timeouts behave. For every error code, see Error codes. For every header, see Headers.

Tell gateway errors from upstream errors

Every response carries an X-Request-Id header, including errors. Include it when you report a problem, because it identifies the request in the gateway’s traces.

The gateway never rewrites an upstream’s error. If the API, LLM provider, or MCP server answers with a 4xx or 5xx, you get its status, headers, and body as it sent them. The exceptions are a response plugin that your admin configured to change responses, and a failover: if the route fails over to another upstream, you only see the last upstream’s answer.

Errors on REST API routes

On a REST route, the gateway’s own errors are short plain-text or JSON bodies, not RFC 9457 problem details. The status code is the reliable signal:

StatusBodyCause
404404 page not foundNo route matches the path.
405Method Not Allowed, with an Allow headerThe path exists, but not for this method.
401Unauthorized, or JSON such as {"error":"invalid or expired token"}The gateway key or token is missing or wrong.
403JSON such as {"error":"access_denied"}You’re authenticated but not allowed to use this route.
422{"error":"validation_failed"}The request failed a validation rule on the route.
429Too Many RequestsA rate limit was hit. See Rate limits.
502Bad GatewayThe upstream couldn’t be reached, or the network connection failed before it answered.
502policy plugin failure or Policy ErrorA plugin on the route failed or is misconfigured. Ask your admin.
503Service Unavailable, or JSON such as {"error":"service paused"}No healthy upstream was available, or the connection is paused.
504Gateway TimeoutThe upstream didn’t start answering within the connection’s response timeout. See Timeouts.

Errors on LLM routes

On an LLM route, errors from the LLM layer use the provider’s error format, so OpenAI and Anthropic SDKs can parse them. The format follows the endpoint you called:

  • OpenAI-compatible endpoints: {"error":{"message":"…","type":"…","code":null}}
  • Anthropic Messages: {"type":"error","error":{"type":"…","message":"…"}}

The gateway uses this format when, for example, the model isn’t allowed on the route (400, invalid_request_error), the connection is paused (503), or every upstream failed (502), was unavailable (503), or timed out (504).

Some errors that the gateway raises before the LLM layer still use the REST shapes in the table above. These include a missing gateway key (401 with Unauthorized), a request rate limit (429 with Too Many Requests), and a route that doesn’t exist (404). An SDK might report these as a generic error, so check the status code.

When a streamed response fails before the provider answers, you get a normal JSON error, not a stream. If it fails after the first event reached you, the stream stops without a closing event.

Errors on MCP routes

MCP clients read two things: the HTTP status and the JSON-RPC body. The gateway uses both:

  • HTTP errors with no JSON-RPC body, for problems before MCP handling or in the transport: 404 for an unknown path, 401 for a missing or wrong credential, 429 for a rate limit, and 502 or 503 when a forwarded request can’t reach the server.
  • HTTP 400 or 404 with a JSON-RPC error, for protocol problems. A protocol header that disagrees with the body (-32020), an unsupported protocol revision (-32022), or missing or incomplete request parameters (-32602) get 400. A method that the gateway doesn’t serve on your protocol revision gets 404 with -32601.
  • HTTP 200 with a JSON-RPC error, for most problems with a specific method or tool call.

The JSON-RPC errors that you’re most likely to see are the following:

CodeMeaningWhat to do
-32001The server needs your own account. The message contains a link.Connect your account.
-32002Temporarily unavailable, or a quota is used up. Check error.data for the reason.Retry later if data.type is upstream_auth_unavailable or data.retryable is true.
-32004You aren’t allowed to call this tool, or the action needs approval.Ask your admin for access.
-32005The connection is paused.Retry later.
-32006The request failed a validation rule.Fix the request.
-32601The tool doesn’t exist (HTTP 200), or the method isn’t served on your protocol revision (HTTP 404).Check the tool name in the tool list, or the method and protocol revision your client uses.
-32603The tool call failed at the server, or the gateway couldn’t look up a credential.Retry if the call is safe to repeat.

Clients on MCP protocol revision 2026-07-28 can see different codes for some of these errors: -32012 instead of -32002, and -32013 instead of -32042 for a request to connect an account through URL elicitation. Some -32002 errors still use -32002 on that revision, so treat both codes the same.

A tool that runs but reports a problem returns a normal result with isError: true. That comes from the MCP server, not the gateway.

Decide whether to retry

The gateway already retries and fails over for you when your admin configures it on the route. For details, see Fail over across providers. Once the gateway starts sending a response to you, it doesn’t retry. Use the following table for your own retries:

ResponseRetry?
429 with Retry-AfterYes, after the number of seconds in Retry-After.
429 without Retry-AfterYes, with exponential backoff.
503Yes, with backoff. If it includes Retry-After, wait that long.
503 with Rate limit not appliedNo. The rate limit counts per caller and the request didn’t identify one, so it has no Retry-After. Authenticate the request first.
502 or 504 from the upstreamOnly if the request is safe to repeat, such as a GET or an LLM request that you don’t mind running twice. The upstream might have received the first request.
502 with policy plugin failure or Policy ErrorNot usually. It’s a configuration problem on the route. Ask your admin if it persists.
400, 401, 403, 404, 405, 413, 422No. Change the request, the credential, or your access first.
A stream that stops partwayRetry the whole request if it’s safe to repeat. The gateway never resumes a stream.

If you disconnect before the gateway answers, the gateway stops the request, records it with status 499, and doesn’t retry it.

Rate limits

A rate limit on the route answers 429 Too Many Requests when you’ve used up your requests for the current window. Whether you get rate-limit headers depends on how your admin set up the limit:

  • With headers turned on, the 429 carries Retry-After and the RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset, RateLimit, and RateLimit-Policy headers, and the same three counters with an X- prefix. Successful requests carry the same headers, without Retry-After, for the tightest limit that applied.
  • With headers turned off, the 429 has no headers that say when to retry. Back off and retry.

An LLM token budget works differently. It counts tokens, not requests, and always answers with Retry-After and a JSON body such as {"error":{"type":"rate_limit_exceeded","message":"Token budget exhausted","retry_after":12}}.

For how admins set limits, see Set rate limits.

Timeouts

Your admin can set timeouts on each connection. Without them, the gateway gives up connecting to an upstream after 15 seconds and on a TLS handshake after 10 seconds, but it doesn’t limit how long the upstream takes to answer. That keeps long LLM responses from being cut off.

When a timeout is set, you see one of the following:

What happenedWhat you get
The upstream didn’t start answering in time504 Gateway Timeout. On an LLM route, it’s in the provider’s error format. On REST and MCP routes, the connection’s overall timeout shows up as 502 Bad Gateway instead. Only the response timeout gives 504 there.
The upstream started answering, then went silentThe response stops partway and the connection closes.
Your client stopped sending the request body408 Request Timeout.

Set a timeout in your own client too, longer than you expect the slowest response to take.

Next steps