Use the OpenAI-compatible API
An LLM route accepts the OpenAI API, so the OpenAI SDK and other OpenAI-compatible clients work with the gateway by changing their base URL. The gateway reads your request to learn the model and whether you asked for a stream, sends the same request to the provider, and relays the provider’s answer. It doesn’t translate between APIs: an OpenAI-format request goes only to a provider endpoint in the OpenAI format. This topic lists the endpoints, shows how to point an SDK at the gateway, and explains streaming, headers, and errors. To send your first request, see Authenticate and send a request.
Prerequisites
Before you begin, make sure you have the following:
- The base URL of an LLM route and a gateway key for it. For details, see Authenticate and send a request.
- A model name that the route serves. For details, see Choose a model.
Find the endpoints
Append the endpoint path to the route’s base URL. The following endpoints use the OpenAI format:
A route serves an endpoint when its provider does. On a model router route, a POST endpoint is available when at least one of the route’s providers serves it, and GET /v1/models is always available. A model router route doesn’t serve /v1/completions, images, audio, files, batches, or paths without the /v1 prefix. A path that the route doesn’t serve gets 404 with unknown LLM endpoint.
Point the OpenAI SDK at the gateway
Set the SDK’s base URL to the route’s base URL, followed by /v1, and send your gateway key in the fg-api-key header. For a complete example in Python and JavaScript, see Use the OpenAI SDK with Fabric AI Gateway.
Some model router routes read the gateway key from the SDK’s own header instead of fg-api-key. On those routes, pass the gateway key as api_key and leave out default_headers. The SDK then sends it as Authorization: Bearer GATEWAY_KEY. Ask your admin which header your route reads. For details, see Accept the gateway key from either SDK.
Stream a response
To stream, set "stream": true in the request. The gateway relays the provider’s server-sent events as they arrive, unchanged. It holds the first event until it’s complete, so that if the provider fails before that event, the gateway can still try another connection.
- Before the stream starts. If the request fails before the provider answers, you get a JSON error, not a stream, as you would from the provider.
- After the first event. The gateway doesn’t retry or fail over. If the connection breaks, the stream ends where it stopped, without a final
[DONE]event.
A streamed chat completion carries token usage only if the request sets "stream_options": {"include_usage": true}. The gateway doesn’t add this option, so without it, the request counts as zero tokens in analytics, budgets, and token rate limits.
Send headers
The gateway passes your headers to the provider, with these exceptions:
- It removes the header that carried your gateway key, and adds the provider’s credential.
- It removes
Accept-Encoding, so responses are never compressed.
Provider headers such as OpenAI-Organization reach the provider unchanged. The response has the provider’s status and headers, plus X-Request-Id. On a model router route, it also has x-fabric-model, which names the <provider>/<model> that answered.
Understand errors
Errors that the gateway writes on an OpenAI-format endpoint use the OpenAI error envelope, so the SDK can parse them:
The response has Content-Type: application/json. Errors from the provider come back exactly as the provider sent them.
Some errors from other plugins aren’t in this envelope. A 401 from Key Auth is plain text, Unauthorized or conflicting credentials, and other plugin errors, such as a 429 from a rate limit, can be plain text too. Check the status code before you parse the body. For authentication errors, see Troubleshoot authentication errors.
On a model router route, a Responses request that carries previous_response_id goes only to the first connection the gateway picks, with no failover, because the stored response lives at the provider account that created it. To keep failover, send the full input with "store": false.
Next steps
- Choose a model
- Use the Anthropic Messages API
- Errors and retries
- For the model error messages, see Fix model errors.

