> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Run the tool-search embedder

> Understand the embedder the data plane chart deploys for MCP tool search, and keep its model cache, scaling, drains, and network access under control.

This page applies to the Enterprise plan, where you run the data plane in your own VPC. The data plane chart deploys an embedder next to the gateway: a small embedding service that turns MCP tool descriptions and search queries into vectors for the [Tool Search](/reference/plugins/tool-search) plugin. This page explains when you need it, and how to keep its model cache, replicas, node drains, and network access the way you want them.

## What the embedder is

The embedder runs HuggingFace [text-embeddings-inference](https://github.com/huggingface/text-embeddings-inference) (TEI) as its own Deployment and Service, both named `fabric-gateway-embedder`. By default it serves the `BAAI/bge-small-en-v1.5` model, which returns 384-dimension vectors, on a CPU image.

The Service name is fixed. It doesn't change with your release or chart name, because the gateway and the collector sidecar find the embedder by that name.

| Default                                     | Value                                                      |
| ------------------------------------------- | ---------------------------------------------------------- |
| Model (`embedder.model.id`)                 | `BAAI/bge-small-en-v1.5`                                   |
| Image (`embedder.image.repository`, `.tag`) | `ghcr.io/huggingface/text-embeddings-inference`, `cpu-1.9` |
| Replicas (`embedder.replicaCount`)          | `1`                                                        |
| Requests and limits (`embedder.resources`)  | 500m CPU and 1Gi memory, up to 2 CPU and 2Gi               |
| Service                                     | Port `80`, forwarding to container port `8080`             |
| Model cache                                 | An `emptyDir` at `/data`                                   |

## When you need it

You need the embedder only if a route uses the Tool Search plugin with the embedder as its embedding service. Tool search needs an OpenAI-compatible embedding service, and the embedder is one that runs inside your cluster, so tool descriptions and queries don't leave it. If you don't use tool search, the embedder sits idle.

To use it, set the Tool Search plugin's embedder to the embedder Service. For example, in the namespace `pm-fabric-dp`:

```json
"embedder": {
  "base_url": "http://fabric-gateway-embedder.pm-fabric-dp.svc.cluster.local/v1",
  "model": "BAAI/bge-small-en-v1.5",
  "dimension": 384
}
```

Keep `/v1` at the end of `base_url`. The gateway sends embedding requests to `<base_url>/embeddings`. The embedder has a private address inside your cluster, which the gateway reaches by default. If you turn on `FABRIC_GW_SSRF_BLOCK_PRIVATE_UPSTREAMS`, add the embedder's host to `FABRIC_GW_SSRF_ALLOW_PRIVATE_UPSTREAM_HOSTS`. See [Calls to private hosts](/run/network#calls-to-private-hosts).

To point every Tool Search instance on the data plane at the same embedder, set `FABRIC_GW_TOOL_SEARCH_EMBEDDER_BASE_URL` under `env.vars` instead. It replaces the `base_url` in each instance's configuration. The chart doesn't set it.

If the embedder is down or too slow, a tool search returns an error rather than an empty result. See [Troubleshoot the data plane](/run/troubleshooting#embedder).

## Keep the model cache across restarts

By default, the model cache is an `emptyDir`, so every embedder pod downloads the model from `huggingface.co` each time it starts. To keep the model on a PersistentVolumeClaim (PVC) instead, so restarts are fast and need no egress once the cache is warm:

```bash
helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
  --set embedder.persistence.enabled=true \
  --set embedder.persistence.size=5Gi
```

The chart creates a PVC named `fabric-gateway-embedder-model-cache` and mounts it at `/data`. Set `embedder.persistence.storageClass` to choose a StorageClass, or `embedder.persistence.existingClaim` to use a PVC you already own. The volume is `5Gi` by default (`embedder.persistence.size`), and `embedder.persistence.annotations` adds annotations to the PVC.

The chart-created PVC is `ReadWriteOnce` by default, which works only with a single embedder replica. The chart refuses to render if you combine it with `embedder.replicaCount` above 1, or with autoscaling and `embedder.autoscaling.maxReplicas` above 1. For more than one replica, do one of the following:

* Use `ReadWriteMany` storage, such as EFS on EKS:

  ```bash
  helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
    --set embedder.persistence.enabled=true \
    --set embedder.persistence.storageClass=efs-sc \
    --set 'embedder.persistence.accessModes[0]=ReadWriteMany'
  ```

* Set `embedder.persistence.existingClaim` to a PVC that supports your topology. The chart doesn't check an existing claim's access mode.

## Run without internet access

If your cluster can't reach `huggingface.co`, run the embedder offline. `embedder.offline.enabled: true` stops the embedder from contacting `huggingface.co`, but the model must already be available locally, or the pod fails to start. Warm the cache first, then go offline:

1. Turn on the persistent cache, with offline mode still off:

   ```bash
   helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
     --set embedder.persistence.enabled=true
   ```

2. Confirm that the model loaded. Look for a `Ready` line:

   ```bash
   kubectl -n pm-fabric-dp logs deploy/fabric-gateway-embedder | grep -i ready
   ```

3. Turn on offline mode, keeping persistence on:

   ```bash
   helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
     --set embedder.persistence.enabled=true \
     --set embedder.offline.enabled=true
   ```

Instead of warming the cache, you can also bake the model into an image you mirror and set `embedder.model.id` to its local directory, or pre-load the cache volume with an init container in `embedder.initContainers`. Mirror the embedder image too, and set `embedder.image.imagePullSecrets` if your registry needs credentials.

## Scale the embedder

Scale the embedder with a static count or with its own HorizontalPodAutoscaler, which needs metrics-server:

```bash
# Static
helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
  --set embedder.replicaCount=3

# Autoscaled
helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
  --set embedder.autoscaling.enabled=true \
  --set embedder.autoscaling.minReplicas=2 \
  --set embedder.autoscaling.maxReplicas=4
```

The autoscaler targets 60% CPU and 75% memory by default (`embedder.autoscaling.targetCPUUtilizationPercentage` and `targetMemoryUtilizationPercentage`), with 1 to 3 replicas. While it's on, the chart ignores `embedder.replicaCount`.

If the embedder can ever run more than one replica and you use a persistent cache, the cache needs `ReadWriteMany` storage or an existing claim. See [Keep the model cache across restarts](#keep-the-model-cache-across-restarts).

To tune throughput, set `embedder.args.maxConcurrentRequests` (default `64`) and `embedder.args.maxBatchTokens` (default `16384`). Any other TEI flag goes in `embedder.extraArgs`. Leave `embedder.args.dtype`, `pooling`, and `autoTruncate` empty to let the embedder detect them from the model. Set `embedder.extraEnv` for variables such as `HF_TOKEN`.

## Keep node drains working

The embedder has its own PodDisruptionBudget, on by default with `minAvailable: 1`. With the default single replica, that budget blocks eviction of the only embedder pod, so `kubectl drain` on its node waits forever. Do one of the following:

* Keep one replica and allow it to be evicted:

  ```bash
  helm upgrade fabric-dp ./fabric-gateway-chart -n pm-fabric-dp -f my-values.yaml \
    --set embedder.podDisruptionBudget.maxUnavailable=1
  ```

  For the embedder, `maxUnavailable` takes precedence over `minAvailable`, so you don't need to remove the default. Tool search is unavailable while the pod moves.

* Run two or more replicas, statically or with autoscaling, so the budget always has a second pod.

## Restrict network access

On a cluster with a NetworkPolicy controller, set `embedder.networkPolicy.enabled: true` to let only the gateway pods reach the embedder. That includes the collector sidecar's metrics scrape. To allow other clients, add them to `embedder.networkPolicy.extraIngress`.

## Security defaults

The embedder runs with the same posture as the gateway: as a non-root user (UID `1000`), with the `RuntimeDefault` seccomp profile, all Linux capabilities dropped, no privilege escalation, and a read-only root filesystem. Its only writable paths are the model cache at `/data` and `/tmp`. If you supply a custom `embedder.image` that writes anywhere else, mount that path as a volume or relax `embedder.securityContext.readOnlyRootFilesystem`.

## Next steps

* To configure tool search on a route, see [Tool Search](/reference/plugins/tool-search).
* To watch the embedder's metrics, see [Monitor the data plane](/run/monitor#metrics).
* To fix embedder problems, see [Troubleshoot the data plane](/run/troubleshooting#embedder).