Skip to navigation

Fabric AI Gateway gives you three ways to control spend:

  • Spending budgets cap what your organization, or one namespace, spends on LLM providers, in US dollars.
  • Billing limits cap what you pay Fabric: request balances and an overage limit.
  • Token budgets cap how many LLM tokens each caller can use.

This topic shows you how to set each one and what callers get when a limit is reached.

Prerequisites

Before you begin, make sure you have the following:

  • For spending budgets, the Manage access control permission to open the organization’s Budget page. To change a budget, you also need permission to manage plugins. On a namespace, a Namespace admin can change that namespace’s budget.
  • For billing limits, the Manage access control permission. Usage & Billing appears under Admin only for organizations billed through the console, not for Enterprise plans.
  • For token budgets, permission to create plugin instances.

Understand spending budgets

A spending budget has a limit in US dollars and a period, either weekly or monthly. Fabric adds up what it can price, and compares the total with the limit. A request with an unknown model or missing token counts is logged and not counted. Periods follow UTC calendar weeks and months, so a monthly budget resets at 00:00 UTC on the first of the month.

You can set a budget at two levels:

  • Organization budget: one limit for the whole organization. Every namespace shares it.
  • Namespace budget: a limit for one namespace. If a namespace has no budget, its spending counts toward the organization budget.

If both apply to a request, the request is refused when either budget is reached. For namespaces, see Manage namespaces.

Fabric works out the cost itself, because providers don’t return a price with the response. It prices each request from a rate card for the provider and model, using the tokens in the response: input, output, reasoning, and cache tokens, and the service tier, such as standard or fast. The result is an estimate, because providers bill you later.

Accounting runs after the response is sent, so it doesn’t slow requests. Spending on the Budget page can lag a little, and the page shows when it last updated.

Set a budget

To set the organization budget, do the following:

  1. In the sidebar, select Admin, and then select Budget.
  2. Select Set budget.

To set a namespace budget, do the following:

  1. Switch to the namespace. For the steps, see Switch namespaces.
  2. In the sidebar, select Budget, and then select Set namespace budget.

You can also open a namespace from Admin > Namespaces and select its Budgets tab.

In the dialog, set the following:

FieldDescription
Spending limit (USD)A positive amount, with up to 6 decimal places.
Budget periodWeekly or Monthly. The default is Monthly.
At budget limitWhat the gateway does when spending reaches the limit. Block requests is the default. Allow requests tracks spending but never refuses a request.

Select Save changes. If you’re creating a budget partway through a period, Fabric counts the spending it already tracked in that period toward the new budget. When there’s no matching usage, the dialog says that the budget will start with new tracking. Until you set a namespace budget, the Budget page shows Spending unavailable and says that spending counts toward the organization budget. To change a budget, select Edit budget. To remove it, select Remove budget, and then select Confirm remove.

Set allowances

On the organization Budget page, Namespace spending lets you set a Default allowance (USD) for each namespace, with up to 100 exceptions. On a namespace’s Budget page, Credential spending does the same for each gateway credential. For other scopes, open the Budget options menu, select Advanced budgets, and create an LLM budget plugin instance, where you can turn on Budget per gateway credential.

Choose what happens at the limit

With Block requests, the gateway checks the spending so far before each request. When it has reached the limit, the gateway refuses the request. The request that takes spending over the limit still completes, so spending can end slightly above the limit.

With Allow requests, the budget only tracks spending. The gateway never refuses a request because of it, and it keeps serving requests even if its spending data is unavailable. The Budget page shows Over budget · Tracking only once spending passes the limit.

Fabric doesn’t yet notify you when spending nears or reaches a limit, so with Allow requests, check the Budget page yourself. The page marks a budget Approaching limit at 80%. At 100% it shows Over budget · Tracking only with Allow requests, and Limit reached with Block requests.

Review spending

The Budget page shows the amount spent, the amount remaining or over, a progress bar, and when the period resets. On the organization page, Spending history charts daily spending for this month, last month, or the last 90 days, and you can compare it with the previous period.

To see the cost of one request, open it in Traces. For details, see Inspect traces.

Add provider discounts

If your organization has a discount from a provider, add it so that budgets use what you really pay. To add a discount, do the following:

  1. In the sidebar, select Admin, and then select Budget.
  2. In Provider discounts, select Add discount.
  3. Choose the Provider, and enter the Discount (%). It must be above 0 and below 100, with up to 2 decimal places.
  4. Select Add discount.

Fabric lowers the cost of that provider’s requests by the percentage when it calculates budget usage. To change or remove a discount, select the pencil icon in its row. If you don’t see Provider discounts, ask an admin for access.

Understand request balances

Fabric counts requests by connection type: API, MCP, and LLM. Each type has a request balance, made up of the requests included in your plan and any requests you buy. The Overview tab of Usage & Billing shows each balance.

When a balance reaches zero, what happens depends on overage:

  • Overage off: The gateway refuses requests of that connection type. Other types keep working.
  • Overage on: Requests keep working, and additional usage is billed at the rates shown on the Pricing tab.
  • Overage on, with a monthly overage limit: When your overage spend for the month reaches the limit, the gateway refuses requests on every billed connection type.

Set an overage limit

To control overage, do the following:

  1. In the sidebar, select Admin > Usage & Billing, and then select the Pricing tab.
  2. Turn on Enable overage to keep connections running when their request balance reaches zero.
  3. In Monthly overage limit, enter a limit in USD, and then select Save limit. Leave the field blank for no limit. A limit of 0 means you pay for nothing beyond the requests you already have.

To stop all overage, turn off Enable overage.

Buy more requests

To add requests to a balance, on the Pricing tab, under Buy more requests, choose the Meter and the Amount, and then select the buy button. You need a payment method on file and a billable connection first.

Cap token use per caller

To limit how many LLM tokens each caller can use, add an LLM Rate Limit instance to a route, connection, or consumer. Set Window (seconds) to the budget period, for example 86400 for a daily budget, and Max tokens to the budget. For the steps and how the count works, see Limit requests and tokens.

A token budget counts tokens, not money. To keep a team’s spend down, use a spending budget, or combine a token budget with model restrictions so the team can use only the models you choose. See Restrict model and tool access.

What callers get when a limit is reached

Spending budget: In Block requests mode, the gateway refuses the request with 429 Too Many Requests and doesn’t send it to the provider. The body is JSON, and there’s no Retry-After header, because the limit doesn’t lift until the period resets or someone raises it:

{ "error": { "type": "llm_budget", "message": "LLM budget exhausted" } }

In Block requests mode, the gateway refuses the request with 503 when it can’t read the current spending, when there’s no active LLM Observability instance, when the budget configuration is invalid, or when accounting is unavailable. In Allow requests mode, it never refuses a request.

Billing limit: The gateway refuses the request with 402 Payment Required and doesn’t send it to the provider. On an LLM route, the error uses the provider’s error format. On an MCP route, the client gets HTTP 200 OK with a JSON-RPC error in the body instead of a 402, so MCP clients can read it. The error includes a reason code and a message:

ReasonMessage
balance_exhaustedrequest allowance exhausted; buy more credit to continue
overage_limitspend limit reached
trial_cap_reachedtrial request allowance exhausted
trial_expiredtrial ended; add a payment card to continue
payment_method_requiredpayment card required
past_duepayment failed; update your payment card to continue
canceledbilling cancelled; resume billing to continue
credits_exhaustedcredit balance exhausted; top up to continue

After you buy requests, raise the limit, or update your payment method, the gateway accepts requests again without any change on the client side.

Token budget: The gateway refuses the request with 429 Too Many Requests and a Retry-After header. For details, see Limit requests and tokens.

A spending budget can also be attached through an LLM budget plugin instance at a chosen scope. See Set allowances.

Verify your budgets

  • On the Budget page, check the amount spent against the limit.
  • On the Overview tab of Usage & Billing, check each request balance.
  • To see which consumers and models use the most tokens and cost the most, see Analytics.

Next steps