Connect an LLM provider

A connection links Fabric AI Gateway to an LLM provider and exposes the provider through a route. A setup wizard creates the service, the credentials, and the route for you. This topic shows you how to create a connection, using Anthropic as the example.

Prerequisites

Before you begin, make sure you have the following:

• Access to a Fabric AI Gateway organization, with permission to create connections.
• An API key for the provider you want to connect.

Create a connection

The setup wizard has four steps: Create service, Set up credentials, Model filter, and Route & gateway. To create a connection, do the following:

1. In the sidebar, select LLMs > Providers. To review your existing connections first, see Manage connections and routes.

2. Next to Anthropic, click Connect. The Use Anthropic page opens.

3. In Create service, enter the following values, and then click Continue:

   Field	Value	Notes
   Service name	anthropic	Use lowercase letters, numbers, and hyphens only. Routes and credentials use this name to refer to the service.
   Target URL	https://api.anthropic.com	The upstream base URL for the provider.
   Description	Optional	The wizard pre-fills a description of the Anthropic Messages API.

4. In Set up credentials, enter the provider’s API key, and then continue. In Model filter, choose which models the connection allows. To allow all current and future models, select that option.

5. In Route & gateway, click Create new route. To use an existing route instead, click Connect to existing route. Then enter the following values:

   Field	Value	Notes
   Route name	anthropic-route
   API compatibility class	Anthropic Messages API	All services on a route must implement the same API type.
   Route base path	/anthropic	Fabric uses this path as the base for the endpoints you expose.
   Endpoints to expose	POST /v1/messages, POST /v1/messages/count_tokens, and GET /v1/models	The wizard selects these endpoints. Claude Code needs only these three. Use Filter by path to narrow the list.
   Access mode	Open	Select Gated if callers need a grant to use the route.

6. Keep Create a gateway credential selected. Fabric generates a credential ID and, in the last step, adds a Key Auth plugin to the route, so callers must authenticate with a gateway key. Clear the checkbox if you already have a credential.

7. For Auth mode, select Gateway key. Select JWT if you authenticate callers through an identity provider.

8. For Credential owner, choose who the key belongs to, for example Org / tenant.

9. Click Finish setup.

Connect a provider with an OpenAI-compatible API

Providers that use an OpenAI-compatible API follow the same steps. The values in Route & gateway differ:

• For API compatibility class, select OpenAI-compatible API.
• For Route base path, enter a path that identifies the provider, for example /my-provider.
• For Endpoints to expose, keep the endpoints that the wizard lists for the provider.

Connect a model that you host

If your security requirements don’t allow traffic to leave your infrastructure, connect a model that you host yourself. When you create the connection, enter your own endpoint as the Target URL. The gateway then routes requests to your infrastructure, and no data goes to another provider.

Verify the connection

To confirm that the wizard created everything, check the following:

1. On LLMs > Connections, find the connection card for the provider.
2. On LLMs > Routes, find the route. The row shows the route’s paths and the plugin instances attached to it.

If you created a gateway credential, the route already has a Key Auth plugin instance. To change how it reads keys, see Authenticate requests with Key Auth.

Next steps

• Send a test request to the route with Postman or a cURL command before you point an application at it. For the steps, see the Quickstart.
• Choose which model handles each request with a routing plugin.
• Connect more providers. Each provider needs its own service, and you can attach services of the same API compatibility class to one route.