> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Limit requests and tokens

> Cap how many requests or LLM tokens a caller can use in a time window, and what clients get when they reach the limit.

Rate limits protect your providers and your spend from a single caller using too much. Fabric AI Gateway has two rate-limiting plugins in the **Traffic** category. **Rate Limit** counts requests on any route. **LLM Rate Limit** counts the LLM tokens that callers use. This topic shows you how to configure each one and what a client gets when it reaches a limit.

## Prerequisites

Before you begin, make sure you have the following:

* A route with authentication, such as Key Auth, so the gateway can tell callers apart. For the steps, see [Authenticate requests with Key Auth](/configure/authentication/key-auth).
* Permission to create plugin instances.

## Limit requests per caller

The Rate Limit plugin limits requests with a throttling algorithm (GCRA, the generic cell rate algorithm) instead of counting them in a rolling window. It lets a caller send up to the **Limit** plus one request at once, and then makes room again at an even rate, which is the **Limit** divided by the window. It refuses requests once the caller is out of room. It attaches to a route only.

To create a Rate Limit instance, do the following:

1. Select **Plugins**, find **Rate Limit** in the **Traffic** category, and select **Use**.

2. Enter a **Name**, for example `rate-limit-chat`. Set **Scope kind** to `route` and **Scope name** to the route.

3. Configure the following settings:

   | Field                | Description                                                                                                                                                                                                                                                                                                                                                                                                                       |
   | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | **Count by**         | What each counter belongs to. **Consumer** keeps one counter per authenticated caller and needs authentication on the route. A request with no value for the chosen **Count by** is refused with `503`, not served unlimited. **Credential** keeps one counter per API key, and only Key Auth provides one. **Global** keeps one counter for every request on the route and is the only option that works without authentication. |
   | **Limit**            | Requests allowed per window, from 1 to 1,000,000,000. Required.                                                                                                                                                                                                                                                                                                                                                                   |
   | **Window (seconds)** | Length of the window that the limit is spread over, from 1 to 31,536,000 (one year). Defaults to 60.                                                                                                                                                                                                                                                                                                                              |

4. Select **Save**.

A route can have one Rate Limit instance for each **Count by** value. For example, you can limit each consumer to 100 requests a minute and the whole route to 5,000 requests a minute. Every instance counts each request, and the first one that's out of room refuses it.

Counts carry over when you change **Limit** or **Window (seconds)**. You can't rename an instance after you create it. Changing **Count by** or renaming the route starts every counter from zero.

## Limit LLM tokens per caller

The LLM Rate Limit plugin gives each caller a token budget per window. After each LLM response, it adds the tokens that the model used to the caller's count. Before each request, it checks the tokens already used in the window and refuses the request if the count has reached the budget. It counts only LLM requests and lets other traffic through.

You can attach LLM Rate Limit to a connection, a route, or a consumer. If more than one instance applies to a request, only the one at the narrowest scope runs, so a consumer limit replaces a route limit for that consumer.

To create an LLM Rate Limit instance, do the following:

1. Select **Plugins**, find **LLM Rate Limit** in the **Traffic** category, and select **Use**.

2. Enter a **Name**, and set **Scope kind** and **Scope name** to the connection, route, or consumer that you want to limit.

3. Configure the following settings:

   | Field                | Description                                                                         |
   | -------------------- | ----------------------------------------------------------------------------------- |
   | **Max tokens**       | The token budget per window, counting input and output tokens. Defaults to 100,000. |
   | **Window (seconds)** | Length of the sliding window. Defaults to 60.                                       |

4. Select **Save**.

Each caller has a separate count on each route. A consumer with a budget of 100,000 tokens can use 100,000 tokens on each route they call. All anonymous callers of a route share one count.

The plugin counts the tokens that the provider reports. If the provider reports none, the request isn't counted. For OpenAI streamed chat completions, that means the client must set `"stream_options": {"include_usage": true}`.

LLM Rate Limit charges the tokens that the [LLM Observability](/reference/plugins/llm-observability) plugin records, so your organization needs a global LLM Observability instance. The control plane doesn't create one for you, and without it the budget never refuses a request. After an Adaptive Model Router cascade, it charges the total of every attempt, whether or not LLM Observability is present.

## Responses when a limit is reached

When a request is refused, the client gets `429 Too Many Requests`, and the gateway doesn't send the request to the provider. A refused request doesn't count against the limit.

* **Rate Limit:** The response carries `Retry-After` and the limit headers only when the instance's `show_headers` setting is on. The console form doesn't offer this setting yet. For the setting, see the [Rate Limit reference](/reference/plugins/rate-limit).

* **LLM Rate Limit:** The response always carries `Retry-After`, the number of seconds until the caller's token count falls below the budget.

Clients should wait for the `Retry-After` interval before they retry. For more about handling errors, see [Handle errors and retries](/use/errors-and-retries).

> **Note**
>
> If you run your own data plane with several replicas, each replica keeps its own counts unless the data plane uses a shared Redis counter store. Without one, a caller can use up to the limit on each replica. For how to set it up, see [Scale and high availability](/run/scale-and-high-availability).

## Verify the limit

Send requests to the route until you reach the limit. The next request returns `429`. With LLM Rate Limit, check the caller's token usage in [Analytics](/use/monitor/analytics).

## Next steps

* [Control spend with budgets](/configure/access-and-limits/budgets)
* [Restrict model and tool access](/configure/access-and-limits/model-access)
* [Scale and high availability](/run/scale-and-high-availability)