Handle errors and retries
A failed request through Fabric AI Gateway can fail in the gateway or in the upstream behind it. When the upstream answers with an error, the gateway passes it to you unchanged. When the gateway itself stops the request, the response shape depends on the route’s protocol and on which step stopped it. This topic explains how errors look on each protocol, which ones are safe to retry, and how rate limits and timeouts behave. For every error code, see Error codes. For every header, see Headers.
Tell gateway errors from upstream errors
Every response carries an X-Request-Id header, including errors. Include it when you report a problem, because it identifies the request in the gateway’s traces.
The gateway never rewrites an upstream’s error. If the API, LLM provider, or MCP server answers with a 4xx or 5xx, you get its status, headers, and body as it sent them. The exceptions are a response plugin that your admin configured to change responses, and a failover: if the route fails over to another upstream, you only see the last upstream’s answer.
Errors on REST API routes
On a REST route, the gateway’s own errors are short plain-text or JSON bodies, not RFC 9457 problem details. The status code is the reliable signal:
Errors on LLM routes
On an LLM route, errors from the LLM layer use the provider’s error format, so OpenAI and Anthropic SDKs can parse them. The format follows the endpoint you called:
- OpenAI-compatible endpoints:
{"error":{"message":"…","type":"…","code":null}} - Anthropic Messages:
{"type":"error","error":{"type":"…","message":"…"}}
The gateway uses this format when, for example, the model isn’t allowed on the route (400, invalid_request_error), the connection is paused (503), or every upstream failed (502), was unavailable (503), or timed out (504).
Some errors that the gateway raises before the LLM layer still use the REST shapes in the table above. These include a missing gateway key (401 with Unauthorized), a request rate limit (429 with Too Many Requests), and a route that doesn’t exist (404). An SDK might report these as a generic error, so check the status code.
When a streamed response fails before the provider answers, you get a normal JSON error, not a stream. If it fails after the first event reached you, the stream stops without a closing event.
Errors on MCP routes
MCP clients read two things: the HTTP status and the JSON-RPC body. The gateway uses both:
- HTTP errors with no JSON-RPC body, for problems before MCP handling or in the transport:
404for an unknown path,401for a missing or wrong credential,429for a rate limit, and502or503when a forwarded request can’t reach the server. - HTTP
400or404with a JSON-RPC error, for protocol problems. A protocol header that disagrees with the body (-32020), an unsupported protocol revision (-32022), or missing or incomplete request parameters (-32602) get400. A method that the gateway doesn’t serve on your protocol revision gets404with-32601. - HTTP
200with a JSON-RPC error, for most problems with a specific method or tool call.
The JSON-RPC errors that you’re most likely to see are the following:
Clients on MCP protocol revision 2026-07-28 can see different codes for some of these errors: -32012 instead of -32002, and -32013 instead of -32042 for a request to connect an account through URL elicitation. Some -32002 errors still use -32002 on that revision, so treat both codes the same.
A tool that runs but reports a problem returns a normal result with isError: true. That comes from the MCP server, not the gateway.
Decide whether to retry
The gateway already retries and fails over for you when your admin configures it on the route. For details, see Fail over across providers. Once the gateway starts sending a response to you, it doesn’t retry. Use the following table for your own retries:
If you disconnect before the gateway answers, the gateway stops the request, records it with status 499, and doesn’t retry it.
Rate limits
A rate limit on the route answers 429 Too Many Requests when you’ve used up your requests for the current window. Whether you get rate-limit headers depends on how your admin set up the limit:
- With headers turned on, the
429carriesRetry-Afterand theRateLimit-Limit,RateLimit-Remaining,RateLimit-Reset,RateLimit, andRateLimit-Policyheaders, and the same three counters with anX-prefix. Successful requests carry the same headers, withoutRetry-After, for the tightest limit that applied. - With headers turned off, the
429has no headers that say when to retry. Back off and retry.
An LLM token budget works differently. It counts tokens, not requests, and always answers with Retry-After and a JSON body such as {"error":{"type":"rate_limit_exceeded","message":"Token budget exhausted","retry_after":12}}.
For how admins set limits, see Set rate limits.
Timeouts
Your admin can set timeouts on each connection. Without them, the gateway gives up connecting to an upstream after 15 seconds and on a TLS handshake after 10 seconds, but it doesn’t limit how long the upstream takes to answer. That keeps long LLM responses from being cut off.
When a timeout is set, you see one of the following:
Set a timeout in your own client too, longer than you expect the slowest response to take.
Next steps
- For problems by symptom, see Troubleshoot Fabric AI Gateway.
- For every error code and body, see Error codes.
- For every header the gateway reads and sets, see Headers.
- To find a failed request, see Inspect traces.
- To connect your account to an MCP server, see Connect your accounts to MCP servers.

