Connect an OpenAI-compatible or self-hosted model server
Many providers and model servers implement the OpenAI API or the Anthropic Messages API, for example vLLM, Ollama, and hosted inference services. You connect them with a manual connection: you pick the API format, enter the server’s URL, and choose how the gateway authenticates. The gateway forwards each request unchanged, apart from the credentials. This topic lists what’s specific to manual LLM connections. For the general setup steps, see Connect an LLM provider.
Choose the API format
When you create the connection, select Manual configuration as the provider, and then select an API format:
Pick the format that the server actually implements. The gateway doesn’t translate between formats, and it reads token usage from the response in the format you pick.
Target URL
Enter the server’s base URL, without the endpoint path. The gateway adds the endpoint path, for example /v1/chat/completions. If the base URL already ends in /v1, the gateway doesn’t repeat it, so https://llm.example.com/v1 and https://llm.example.com both work.
For a model that you host, the data plane must be able to reach the server:
- On the Enterprise plan, the data plane runs in your VPC, so it can reach servers on your private network.
- On the Free plan, the data plane runs in Postman’s cloud, so the server must be reachable from the internet.
By default, the data plane connects to private addresses, so you don’t need to allow-list a server on your network. It never connects to loopback addresses such as localhost, or to cloud metadata addresses. Use a hostname or address that the data plane can reach.
Credentials
Choose an Auth type for the gateway to use when it calls the server:
The gateway replaces a client header that has the same name as the credential header. Callers authenticate to the gateway with their gateway key in fg-api-key.
Model names
Use the names that the server expects, for example meta-llama/Llama-3.1-8B-Instruct for a vLLM server. The gateway sends the model field exactly as the client wrote it. The console’s model picker lists only catalog models, so to restrict a connection like this one to specific names, set allowed_models through the admin API.
Example
This request goes to a route whose connection points at an OpenAI-compatible server:
Replace BASE_URL with the route’s base URL and GATEWAY_KEY with your gateway key. For how to find both, see Authenticate and send a request.

