Connect an LLM provider
A connection links Fabric AI Gateway to an LLM provider and exposes the provider through a route. One form creates the connection, stores the provider credential, and creates the route. This topic shows you how to create a connection, using Anthropic as the example, and how to create a gateway key for callers.
Prerequisites
Before you begin, make sure you have the following:
- Access to a Fabric AI Gateway organization, with permission to create connections.
- An API key for the provider you want to connect.
Create a connection
The Create LLM connection page has three sections: Connection, Upstream auth, and Gateway route. To create a connection, do the following:
-
In the sidebar, select LLMs, and then select the Providers tab. To review your existing connections first, see Manage connections and routes.
-
Next to Anthropic, select Connect. The Create LLM connection page opens with Anthropic selected as the Provider.
-
In Connection, review the following values:
Fabric fills in the provider’s target URL for you.
-
In Upstream auth, enter the provider’s API key. If your organization keeps its own vault, select the Vault and enter the secret’s name in that vault instead.
-
In Gateway route, keep Create gateway route selected, and then review the following values:
-
Select Create connection. Fabric creates the connection and the route, adds a Key Auth plugin instance to the route, and opens the connection’s page.
Creating the connection doesn’t create a gateway key. To create one, follow Create a gateway key.
The form doesn’t offer JWT authentication. If you run your own data plane and want to authenticate callers with JWTs from your identity provider, add a JWT Auth plugin instance to the route from Plugins. For the steps, see Authenticate requests with JWT Auth.
Create a gateway key
Callers send a gateway key with each request to a route that uses Gateway key. To create a gateway key, do one of the following:
- On the Create LLM connection page, under Gateway auth, select Create key.
- In the sidebar, select Credentials, select Create credential, and then select Gateway key as the Type.
Then do the following:
- In Owner, choose who the key belongs to, for example Myself or Organization.
- Optionally, enter a Name for the key.
- Select Create key in the dialog. On the Create credential page, the button is Create gateway key.
- Copy the key from the Gateway key created dialog. The console shows the key only once.
Connect a provider with an OpenAI-compatible API
To connect a provider that isn’t in the catalog but uses an OpenAI-compatible API, do the following:
- In the sidebar, select LLMs, select the Connections tab, and then select Create connection.
- For Provider, select Manual configuration.
- Enter a Connection name and the provider’s Target URL.
- For API format, select OpenAI-compatible API.
- In Upstream auth, choose the Auth type that the provider expects, and enter its key.
- In Gateway route, enter a Base path that identifies the provider, for example
/my-provider, and keep the endpoints that Fabric lists. - Select Create connection.
Connect a model that you host
If your security requirements don’t allow traffic to leave your infrastructure, connect a model that you host yourself. Create the connection with Manual configuration, and enter your own endpoint as the Target URL. The gateway then routes requests to your infrastructure, and no data goes to another provider.
Verify the connection
To confirm that Fabric created everything, check the following:
- In LLMs, on the Connections tab, find the connection card for the provider.
- On the Routes tab, find the route. The row shows the route’s paths and the plugin instances attached to it.
If you selected Gateway key, the route has a Key Auth plugin instance named route-auth-- followed by the route name, for example route-auth--anthropic-route. For more about how it reads keys, see Authenticate requests with Key Auth.
Next steps
- Send a test request to the route with Postman or a cURL command before you point an application at it. For the steps, see Quickstart: Send your first request on the hosted gateway.
- Choose which model handles each request with a routing plugin.
- Connect more providers. Each provider needs its own connection, and you can attach connections that use the same API format to one route.

