> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Use the OpenAI-compatible API

> Call chat completions, responses, embeddings, and the model list through an LLM route, with streaming and OpenAI-shaped errors.

An LLM route accepts the OpenAI API, so the OpenAI SDK and other OpenAI-compatible clients work with the gateway by changing their base URL. The gateway reads your request to learn the model and whether you asked for a stream, sends the same request to the provider, and relays the provider's answer. It doesn't translate between APIs: an OpenAI-format request goes only to a provider endpoint in the OpenAI format. This topic lists the endpoints, shows how to point an SDK at the gateway, and explains streaming, headers, and errors. To send your first request, see [Authenticate and send a request](/use/authenticate).

## Prerequisites

Before you begin, make sure you have the following:

* The base URL of an LLM route and a gateway key for it. For details, see [Authenticate and send a request](/use/authenticate).
* A model name that the route serves. For details, see [Choose a model](/use/llm/choose-a-model).

## Find the endpoints

Append the endpoint path to the route's base URL. The following endpoints use the OpenAI format:

| Method and path             | What it does                                                              |
| --------------------------- | ------------------------------------------------------------------------- |
| `POST /v1/chat/completions` | Chat completions.                                                         |
| `POST /v1/responses`        | Responses.                                                                |
| `POST /v1/embeddings`       | Embeddings.                                                               |
| `GET /v1/models`            | The model list. On a model router route, the gateway answers it.          |
| `GET /v1/models/<model>`    | One model from the list. On a model router route, the gateway answers it. |

A route serves an endpoint when its provider does. On a model router route, a `POST` endpoint is available when at least one of the route's providers serves it, and `GET /v1/models` is always available. A model router route doesn't serve `/v1/completions`, images, audio, files, batches, or paths without the `/v1` prefix. A path that the route doesn't serve gets `404` with `unknown LLM endpoint`.

## Point the OpenAI SDK at the gateway

Set the SDK's base URL to the route's base URL, followed by `/v1`, and send your gateway key in the `fg-api-key` header. For a complete example in Python and JavaScript, see [Use the OpenAI SDK with Fabric AI Gateway](/guides/sdks/openai).

Some model router routes read the gateway key from the SDK's own header instead of `fg-api-key`. On those routes, pass the gateway key as `api_key` and leave out `default_headers`. The SDK then sends it as `Authorization: Bearer GATEWAY_KEY`. Ask your admin which header your route reads. For details, see [Accept the gateway key from either SDK](/configure/routing/model-router#accept-the-gateway-key-from-either-sdk).

## Stream a response

To stream, set `"stream": true` in the request. The gateway relays the provider's server-sent events as they arrive, unchanged. It holds the first event until it's complete, so that if the provider fails before that event, the gateway can still try another connection.

* **Before the stream starts.** If the request fails before the provider answers, you get a JSON error, not a stream, as you would from the provider.
* **After the first event.** The gateway doesn't retry or fail over. If the connection breaks, the stream ends where it stopped, without a final `[DONE]` event.

> **Note**
>
> A streamed chat completion carries token usage only if the request sets `"stream_options": {"include_usage": true}`. The gateway doesn't add this option, so without it, the request counts as zero tokens in analytics, budgets, and token rate limits.

## Send headers

The gateway passes your headers to the provider, with these exceptions:

* It removes the header that carried your gateway key, and adds the provider's credential.
* It removes `Accept-Encoding`, so responses are never compressed.

Provider headers such as `OpenAI-Organization` reach the provider unchanged. The response has the provider's status and headers, plus `X-Request-Id`. On a model router route, it also has `x-fabric-model`, which names the `<provider>/<model>` that answered.

## Understand errors

Errors that the gateway writes on an OpenAI-format endpoint use the OpenAI error envelope, so the SDK can parse them:

```json
{
  "error": {
    "message": "model `openai/gpt-9` not found",
    "type": "invalid_request_error",
    "code": "model_not_found"
  }
}
```

The response has `Content-Type: application/json`. Errors from the provider come back exactly as the provider sent them.

| Status                     | When                                                                         |
| -------------------------- | ---------------------------------------------------------------------------- |
| `400`, `403`, `404`, `413` | The gateway can't serve the model you named.                                 |
| `404`                      | The route doesn't serve the endpoint. The message is `unknown LLM endpoint`. |
| `502`                      | The last connection the gateway tried failed. The `type` is `server_error`.  |
| `503`                      | No connection was available. The `type` is `server_error`.                   |
| `504`                      | The last connection timed out. The `type` is `server_error`.                 |

Some errors from other plugins aren't in this envelope. A `401` from Key Auth is plain text, `Unauthorized` or `conflicting credentials`, and other plugin errors, such as a `429` from a rate limit, can be plain text too. Check the status code before you parse the body. For authentication errors, see [Troubleshoot authentication errors](/configure/authentication/key-auth#troubleshoot-authentication-errors).

> **Note**
>
> On a model router route, a Responses request that carries `previous_response_id` goes only to the first connection the gateway picks, with no failover, because the stored response lives at the provider account that created it. To keep failover, send the full input with `"store": false`.

## Next steps

* [Choose a model](/use/llm/choose-a-model)
* [Use the Anthropic Messages API](/use/llm/anthropic-messages)
* [Errors and retries](/use/errors-and-retries)
* For the model error messages, see [Fix model errors](/use/llm/choose-a-model#fix-model-errors).