Manage gateway connections and routes | Fabric Gateway Docs

Fabric AI Gateway can manage your LLM providers, MCP servers, and plain REST APIs. They share the same underlying connection model — a connection, one or more routes, and an upstream target — but each has settings specific to its protocol.

Learn how to manage LLM connections and routes, configure an MCP server connection, and connect a REST API as a service. If you haven’t connected anything yet, start with the Quickstart.

Connect and manage LLM providers

LLMs has the following components:

Component	What it’s for
Connections	Every LLM provider you’ve connected, such as `anthropic`, `bedrock`, or `openai`.
Routes	The route generated for each connection, including every path it exposes and the plugin instance attached to it.
Providers	The full catalog of LLM providers available to connect in your tenant.

View your connections

To view your LLM connections, do the following:

In the sidebar, select LLMs under Gateway.
Select the Connections tab.

Each connection shows its target URL, a short description, tags, and its upstream count. To add another provider, select Create connection.

View your routes

To view the routes generated for your connections, do the following:

In the sidebar, select LLMs under Gateway.
Select the Routes tab.

Each row shows a route’s paths and methods, along with the plugin instance attached to it. For example:

Route	Paths	Plugin instance
`anthropic`	`/anthropic/v1/messages` (POST), `/anthropic/v1/messages/count_tokens` (ANY), `/anthropic/v1/models` (GET)	`route-auth--anthropic`
`openai`	Chat completions, responses, embeddings, models, audio transcription/speech, image generation, moderations	`route-auth--openai`

Browse available providers

To browse the LLM provider catalog, do the following:

In the sidebar, select LLMs under Gateway.
Select the Providers tab.

This tab lists every LLM provider available to connect in your tenant — 22 providers, in a typical tenant.

Configure an MCP server connection

When Service type is set to MCP, the connection editor exposes tools and resources to agents over MCP (JSON-RPC/SSE), rather than proxying plain REST or LLM traffic.

Configure an MCP connection

To configure an MCP connection, do the following:

In the sidebar, select MCP servers under Gateway, then open the connection you want to configure (for example, linear-mcp).
Under Service type, select MCP.
Enter the Target URL for the MCP server, for example https://mcp.linear.app/mcp.
Under Upstream auth requirement, select an Auth type (for example, Bearer) and set Applies to (for example, Org/tenant).
Optionally, expand Identity binding or Advanced settings to configure identity mapping.
Enter a Description and, optionally, select a Category.
Click Save changes.

Add upstreams

An MCP connection can point to one or more upstreams. Each upstream has its own Target URL, Weight, and Name, along with a Per-upstream OAuth / MCP discovery checkbox.

To add an upstream, do the following:

In the connection editor, click Add upstream.
Enter a Target URL, Weight, and Name for the upstream.
Select a Load balancer strategy.

Weight controls how traffic is split across upstreams, but it’s only used by load balancer strategies that support weighted splitting. Under Round robin, traffic rotates evenly across upstreams and weight is ignored.

Set the access mode

Under Access mode, select one of the following:

Access mode	Behavior
Open	The connection is reachable without additional gating.
Gated	The connection requires gating before it’s reachable.

Connect a REST API as a service

Services connects plain HTTP APIs the same way you connect LLM providers and MCP servers. Add a service, then import an OpenAPI spec to generate its routes automatically.

Import an OpenAPI spec

Import an OpenAPI spec is a 3-step wizard.

Step 1: Provide spec

Paste or upload a JSON/YAML OpenAPI document. Under Import mode, select one of the following:

Import mode	Behavior
Merge	Add and update routes from the spec; leave routes not in the spec untouched.
Overwrite	Match the spec exactly, removing any routes that aren’t in it.

Optionally, expand Advanced settings to set:

Import ID — a stable identifier so re-imports keep updating the same API instead of creating a new one.
Route prefix — mounts every route from the spec under a path prefix, which is stripped before the request is forwarded upstream, so multiple imported APIs don’t collide.
Upstream URL override.

Step 2: Review plan

Review the plan before you publish anything. The plan shows which service will be created or updated, how many routes are new, changed, or unchanged, and the exact route paths and methods.

Imported services don’t get gateway-managed auth by default — callers pass their own credentials through to the upstream unless you add an upstream credential on the Credentials page after publishing. For how to add one, see Manage users, roles, and credentials.

Step 3: Publish

Click Publish to apply the plan.

Re-import to update routes

Re-importing an updated spec in Merge mode applies incremental updates. For example, adding a PUT operation to an existing path and a new path produces a plan showing 1 new, 1 changed, and 3 unchanged routes, with an option to expand and view the unchanged ones.

Routes from multiple imports appear together under Services → Routes — for example, two services imported from the same spec at different times can end up contributing a combined 10 routes across albums, comments, posts (with both GET and PUT), todos, and users.

Next steps

To add authentication, rate limiting, or transforms to any of these routes, see Control and transform gateway traffic with plugins.
To monitor traffic on these routes, see Monitor gateway traffic with analytics and traces.
To add an upstream credential for a connected service, see Administration.