Skip to navigation

Manage connections and routes

Fabric AI Gateway can manage your LLM providers, MCP servers, and plain REST APIs. They share the same underlying connection model: a connection, one or more routes, and an upstream target. Each type also has settings specific to its protocol.

This topic shows you how to choose what to connect and how to review your connections, routes, and providers. If you haven’t connected anything yet, start with Quickstart: Send your first request on the hosted gateway.

Choose what to connect

What you connectUse it toGuide
LLM providerProxy requests to a model provider, such as Anthropic or OpenAI.Connect an LLM provider
MCP serverExpose tools and resources to agents over MCP.Connect an MCP server
REST APIConnect a plain HTTP API by importing an OpenAPI spec.Connect a REST API

Understand the LLMs page

LLMs has the following components:

ComponentWhat it’s for
ConnectionsEvery LLM provider you’ve connected, such as anthropic, bedrock, or openai.
RoutesThe route generated for each connection, including every path it exposes and the plugin instance attached to it.
ProvidersThe full catalog of LLM providers available to connect in your organization.

Each tab shows a count, for example the number of connections.

View your connections

To view your LLM connections, do the following:

  1. In the sidebar, select LLMs.
  2. Click the Connections tab.

Each connection card shows the connection name, an Open or Gated badge, the provider icon, and the target URL. Use Search connections to find a connection. In organizations with many connections, the list spans several pages, and the footer shows the range you’re viewing, for example 1–24 of 165 connections. To add another provider, click Create connection. You can also specify your own target URL, which lets you connect a model that you host yourself.

Open a connection

To open a connection, select it in the Connections tab. The page shows the connection’s name and type, for example LLM · Amazon Bedrock Mantle, and a ⋯ menu. It has the following tabs:

TabWhat it shows
OverviewDescription, Access, and Models, each with Edit. Network shows Public or Private and can’t be edited. Upstreams shows how many upstreams the connection has and how it balances them, for example 1 · Round robin.
UpstreamThe upstream URLs and the balancing method, with Edit. Auth requirement shows what Fabric sends to the upstream and in which header: Auth type (for example Bearer token), Applies to (for example Shared), Header (for example Authorization), and Credential. If no credential exists, select Add credential.
RoutesThe routes for this connection, with an Edit link on each. Select Add route to add one.
PluginsThe plugin instances that apply to the connection.
AccessGrants for a gated connection. The tab is unavailable while Access is Open.

View your routes

To view the routes generated for your connections, do the following:

  1. In the sidebar, select LLMs.
  2. Select the Routes tab.

Each row shows a route’s paths and methods, along with the plugin instance attached to it. For example:

RoutePathsPlugin instance
anthropic-route/anthropic/v1/messages (POST), /anthropic/v1/messages/count_tokens (ANY), /anthropic/v1/models (GET)route-auth--anthropic-route
openai-routeChat completions, responses, embeddings, models, audio transcription and speech, image generation, and moderationsroute-auth--openai-route

Use Search routes to find a route, the All connections filter to show the routes for one connection, and the sort menu to order the list by name. Each row has a copy icon that copies the route’s base URL, and a ⋯ menu with the endpoint actions, such as Copy chat URL (POST) and Copy chat cURL (POST). The menu offers only the endpoints that the route exposes.

Fabric names a new route after its connection, <connection>-route, and names the route’s Key Auth instance route-auth--<route>.

Add a route

A connection gets its first route when you create it. To add another route, for example with different authentication, or one route that serves several connections, do the following:

  1. In the sidebar, select LLMs, MCP servers, or Services, and then select the Routes tab. Select Create route. You can also open a connection, select its Routes tab, and select Add route.
  2. In Route, enter a Name, choose the Protocol, and select one or more Connections.
  3. In Request matching, optionally enter a Base path, for example /provider. To add more conditions, expand Advanced matching.
  4. Keep Require gateway auth selected unless the route serves public endpoints. Fabric sets Auth type to Gateway key and shows the Request header, fg-api-key, which you can’t change. Select Create key if you need a key.
  5. Select Create route.

Browse available providers

To browse the LLM provider catalog, do the following:

  1. In the sidebar, select LLMs.
  2. Select the Providers tab.

This tab lists every LLM provider that you can connect in your organization. Click Connect next to a provider to open the Create LLM connection page.

Set the access mode

Access is a setting on the connection, and it applies to the connection on every route:

Access modeBehavior
OpenAnyone who passes gateway auth can call the connection.
GatedOnly callers with a grant can call the connection. The gateway refuses other callers.

You set Access in the Connection section when you create a connection. To change it later, open the connection, and then on the Overview tab, select Edit next to Access.

To give callers access to a gated connection, open the connection, select the Access tab, and then select Add grant. In Grant access to, choose who the grant covers:

OptionWho gets access
UserOne user.
RouteEvery authenticated caller on one route to the connection.
EveryoneEvery authenticated caller, on every route to the connection.

Next steps