> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Connect an OpenAI-compatible or self-hosted model server

> Connect any server that implements the OpenAI or Anthropic Messages API, including models you host yourself.

Many providers and model servers implement the OpenAI API or the Anthropic Messages API, for example vLLM, Ollama, and hosted inference services. You connect them with a manual connection: you pick the API format, enter the server's URL, and choose how the gateway authenticates. The gateway forwards each request unchanged, apart from the credentials. This topic lists what's specific to manual LLM connections. For the general setup steps, see [Connect an LLM provider](/configure/connections/llm-providers).

## Choose the API format

When you create the connection, select **Manual configuration** as the provider, and then select an **API format**:

| API format                 | Endpoints the route can expose                                                                                         |
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| **OpenAI-compatible API**  | `POST /v1/chat/completions`, `POST /v1/responses`, `POST /v1/completions`, `POST /v1/embeddings`, and `GET /v1/models` |
| **Anthropic Messages API** | `POST /v1/messages`, `POST /v1/messages/count_tokens`, and `GET /v1/models`                                            |

Pick the format that the server actually implements. The gateway doesn't translate between formats, and it reads token usage from the response in the format you pick.

## Target URL

Enter the server's base URL, without the endpoint path. The gateway adds the endpoint path, for example `/v1/chat/completions`. If the base URL already ends in `/v1`, the gateway doesn't repeat it, so `https://llm.example.com/v1` and `https://llm.example.com` both work.

For a model that you host, the data plane must be able to reach the server:

* On the Enterprise plan, the data plane runs in your VPC, so it can reach servers on your private network.
* On the Free plan, the data plane runs in Postman's cloud, so the server must be reachable from the internet.

By default, the data plane connects to private addresses, so you don't need to allow-list a server on your network. It never connects to loopback addresses such as `localhost`, or to cloud metadata addresses. Use a hostname or address that the data plane can reach.

## Credentials

Choose an **Auth type** for the gateway to use when it calls the server:

| Auth type        | What the gateway sends                                                                                                           |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Bearer token** | `Authorization: Bearer <token>`. The default for the OpenAI-compatible format.                                                   |
| **API key**      | The key in the **Header** that you name. The default header is `x-api-key`, which is what the Anthropic Messages format expects. |
| **None**         | No credential. Use this for a server on your own network that doesn't require one.                                               |

The gateway replaces a client header that has the same name as the credential header. Callers authenticate to the gateway with their gateway key in `fg-api-key`.

## Model names

Use the names that the server expects, for example `meta-llama/Llama-3.1-8B-Instruct` for a vLLM server. The gateway sends the `model` field exactly as the client wrote it. The console's model picker lists only catalog models, so to restrict a connection like this one to specific names, set `allowed_models` through the [admin API](/configure/automation/admin-api).

## Example

This request goes to a route whose connection points at an OpenAI-compatible server:

```bash
curl "BASE_URL/v1/chat/completions" \
  -H "fg-api-key: GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{ "role": "user", "content": "Hello" }]
  }'
```

Replace `BASE_URL` with the route's base URL and `GATEWAY_KEY` with your gateway key. For how to find both, see [Authenticate and send a request](/use/authenticate).

## Next steps

* [Plan a deployment](/run/plan-a-deployment)
* [Fail over across providers](/configure/routing/failover)
* [Restrict model and tool access](/configure/access-and-limits/model-access)