Skip to navigation

Route requests through Lightyear

Lightyear is a mode of the LLM Router plugin. When you attach it to an LLM route, requests go to the Lightyear service first and then to the model provider. Lightyear applies cache optimizations and a code mode, with the aim of reducing LLM spend. This topic shows you where to find Lightyear mode, how to turn it on, and how to use it.

Lightyear is experimental and in development. Its behavior can change, and results such as cost savings can vary. Test it on your own workloads before you rely on it.

Prerequisites

Before you begin, make sure you have the following:

  • An LLM route to Anthropic, OpenAI, or Google Gemini that accepts authenticated requests. Gemini works only with Use the client’s provider credentials turned on. For the steps, see Connect an LLM provider.
  • A route with exactly one LLM connection.
  • Permission to create plugin instances.
  • A gateway key for the route.

Understand Lightyear mode

Lightyear isn’t a separate plugin. It’s one of the modes of the LLM Router plugin, alongside single, fallback, and loadbalance. You can attach it to an LLM route whose connection goes to Anthropic, OpenAI, or, with client credentials, Google Gemini. The gateway rejects Lightyear mode for other providers.

Without Lightyear, a request on a route goes straight to the provider that you connected. With Lightyear mode on, the same request goes to the Lightyear service, which forwards it to the provider.

Lightyear is designed for long conversations, such as sessions with a coding agent like Claude Code, where many requests build on each other. A single request sent from a REST client is unlikely to show an effect.

Turn on Lightyear mode

To turn on Lightyear mode, do the following:

  1. Click Plugins, find LLM Router in the Routing category, and select Use.
  2. Enter a Name, for example lightyear-router.
  3. Set Scope kind to route, and set Scope name to the route you want to configure. The LLM Router can be scoped only to a route.
  4. For Mode, select lightyear.
  5. Decide whether to turn on Use the client’s provider credentials. For more information, see the next section.
  6. Click Save.

The route’s row on LLMs > Routes then lists the plugin instance under Plugin instances.

Choose how the route authenticates to the provider

The Use the client’s provider credentials toggle controls whose credentials reach the provider:

SettingBehavior
Off (default)The gateway authenticates to the provider. You don’t send a provider key, and Fabric adds the credential that you saved on the connection.
OnThe request carries its own provider credentials, and the gateway passes them through. A client that’s already signed in to the provider, such as Claude Code, uses its own access token, so you don’t need to enter an API key anywhere. A gateway key is still required. Send it in a separate header such as fg-api-key, not in Authorization, x-api-key, or x-goog-api-key, which carry the provider credential.

If you turn the toggle on and send a request without provider credentials, the provider rejects it. For example, Anthropic returns an authentication_error that says the x-api-key header is required.

Send traffic through the route

After you save the instance, requests on the route use Lightyear. To test with Postman, send a request with your gateway key. If the toggle is on, also add the provider’s key in the provider’s header.

To use Lightyear from a coding agent, point the agent at the route, as the next section shows. With the toggle on, the agent signs in as usual and passes its own credentials through.

Point a coding agent at the route

Coding agents are where Lightyear helps most, because their sessions are long. Select your agent, and add the gateway as its provider. Replace BASE_URL with the base URL of your Lightyear route and GATEWAY_KEY with your gateway key. To get the base URL of a route, select Copy LLM base URL on LLMs > Routes.

Add the following to ~/.claude/settings.json, or to .claude/settings.json in a project, and then restart Claude Code:

{
"env": {
"ANTHROPIC_BASE_URL": "BASE_URL",
"ANTHROPIC_CUSTOM_HEADERS": "fg-api-key: GATEWAY_KEY"
}
}

With Use the client’s provider credentials off, the gateway adds the Anthropic key saved on the connection. With it on, Claude Code sends its own sign-in token, so you don’t set an API key.

For more information, see Use Claude Code with Fabric AI Gateway.

Treat the gateway key as a secret. Don’t commit a settings file that contains it.

Confirm that requests use Lightyear

To confirm that requests go through Lightyear, open Traces. A request on the route shows a request to the Lightyear service, followed by the request to the provider. A request on a route without Lightyear shows only the request to the provider.

Review Lightyear savings

When an enabled LLM Router instance in lightyear mode exists in the namespaces you’re viewing, the LLM tab of Analytics shows a Lightyear savings card. To open it, select Analytics in the sidebar (or Admin > Analytics for the whole organization), and then select the LLM tab. The card estimates what Lightyear saved compared with the same sessions calling the provider directly. It shows the following:

ItemWhat it shows
HeadlineThe estimated saving, and what share of the native cost it is, for example “12% less than native clients”. If Lightyear cost more, it reads “more than native clients”.
Native estimateAn estimate of what the same sessions would have cost without Lightyear.
Actual spendWhat the Lightyear sessions cost.
Lightyear sessionsThe number of sessions, with their generations and users.

If no Lightyear sessions ran in the selected window, the card says No Lightyear sessions in the selected window. The card doesn’t appear when no enabled Lightyear instance exists in the namespaces you’re viewing.

The figure is a heuristic estimate, and it uses standard list prices without your organization’s discounts. Because Lightyear is experimental, treat it as a guide, and compare it with the token usage and cost on the rest of the LLM tab.

Next steps