Data plane production checklist
This page applies to the Enterprise plan, where you run the data plane in your own VPC. Work through this checklist before the data plane carries production traffic. The chart ships with production-oriented defaults, so most items either confirm that you kept a default or ask you to replace a placeholder. Each item links to the page that explains it.
Defaults to keep
The chart turns these on for you. Keep them on.
- Production mode.
FABRIC_GW_PRODUCTION_MODEis"true". The data plane then refuses to start without its encryption key, a tenant ID, or the control plane CA certificate, or without a bootstrap token unless workload attestation is on. It also refuses to start withFABRIC_GW_STORE_AUTO_MIGRATEorFABRIC_GW_DEBUG_SCOPESset to"true", and credential brokering fails closed. Set it to"false"only for local testing. - Pod security. The gateway and embedder containers run as non-root with the
RuntimeDefaultseccomp profile, drop all Linux capabilities, block privilege escalation, and use a read-only root filesystem. The OpenTelemetry Collector sidecar runs as non-root with theRuntimeDefaultprofile, but has no container security context of its own. If your policy requires one, copy the chart’s wholeotel-collectorentry intoadditionalContainersin your values file, and add asecurityContext. Writable state lives only in mounted volumes:/data/cachefor the gateway, and/dataand/tmpfor the embedder. - Health probes. Liveness, readiness, and startup probes all point at
/healthzon port8080. See Monitor the data plane. - Disruption budgets. The gateway and embedder each have a PodDisruptionBudget with
minAvailable: 1. - Tracing. The gateway samples 10% of new traces (
OTEL_TRACES_SAMPLER_ARG: "0.10"), and doesn’t capture LLM messages or MCP tool arguments and results (FABRIC_GW_TRACING_CAPTURE_MESSAGE_CONTENT: "false"). Raise the sample rate or turn on message capture only after you review the cost and your data-handling rules. - Private hosts.
FABRIC_GW_SSRF_ALLOW_PRIVATE_HOSTSis empty, so the gateway’s own authentication calls reach no private host you haven’t listed. The gateway’s authentication calls and upstream calls never reach link-local or cloud metadata addresses. Upstream calls to loopback are refused unless you setFABRIC_GW_SSRF_ALLOW_LOOPBACK_UPSTREAMSto"true".
Install and configuration
-
Every image the chart pulls is mirrored into your registry and pinned:
- The gateway:
image.repository, withimage.digestor an exactimage.tag. - The embedder,
ghcr.io/huggingface/text-embeddings-inference:cpu-1.9:embedder.image.repositoryandembedder.image.digest. - The migration Job,
flyway/flyway:10:migrations.image, pinned to an exact tag. - The collector sidecar,
otel/opentelemetry-collector-contrib:0.149.0: theimageof theotel-collectorentry inadditionalContainers. - OPA, if you turn it on:
opa.image.repositoryandopa.image.tag.
The migration Job doesn’t use
image.imagePullSecrets. If your registry needs credentials, add the pull Secret to the namespace’sdefaultServiceAccount, which the Job runs as. See Install the data plane with Helm. - The gateway:
-
Every placeholder in the chart’s values is replaced:
image.repository,FABRIC_GW_TENANT_ID,FABRIC_GW_CP_ADDRESS,FABRIC_GW_MYSQL_HOST,FABRIC_GW_ADMIN_EXTERNAL_URL,FABRIC_GW_BROKER_EXTERNAL_URL, andotel.clickhouse.*if you use the default collector. See Helm values. -
Your values file is in version control, holds no secret values, and is passed on every
helm upgrade. -
The control plane CA certificate is the one Postman gave you, supplied with
syncCA.pemor thefabric-dp-sync-caSecret. -
FABRIC_GW_ADMIN_EXTERNAL_URLandFABRIC_GW_BROKER_EXTERNAL_URLmatch the external URL clients use, and Admin > Data plane > Base URL in the console is set to the same URL. See Expose the gateway to your clients. -
You decided whether routes may reach upstreams on private addresses. They can by default. To block them, set
FABRIC_GW_SSRF_BLOCK_PRIVATE_UPSTREAMSto"true"and list the internal hosts your routes need inFABRIC_GW_SSRF_ALLOW_PRIVATE_UPSTREAM_HOSTS. Include the embedder’s host from your Tool Search plugin’s embedder URL, or tool search stops working. See Calls to private hosts. -
FABRIC_GW_SSRF_ALLOW_PRIVATE_HOSTSlists only identity provider or OAuth hosts on your private network, if any. -
If you use the access-control, validator, or transformer policy plugins, you know that they need the OPA sidecar (
opa.enabled) and fail closed without it. Before you turn it on, read the readiness warning in Helm values: in chart 0.6.0, the sidecar’s readiness check can’t succeed and pods stay at2/3.
Secrets and identity
- The app Secret is created from your secret manager and named in
secrets.app.existingSecret.secrets.dev.enabledisfalse. See Secrets and keys. FABRIC_GW_SECRET_KEYis backed up somewhere durable and is the same on every pod.- You have a plan for new pods to enroll: workload attestation (recommended), or a bootstrap token that you mint in the console when you add or replace pods. A token lasts 10 minutes by default and 30 minutes at most. See Connect the data plane to the control plane.
- Bootstrap tokens are stored like secrets, and the people who can mint them are limited.
- You know how to revoke a node’s certificate from Admin > Gateway health, and that you also need to delete the pod to stop its traffic.
Network
- Egress is allowed to the control plane sync address, your secret store, your telemetry backend, your upstreams, and
huggingface.counless the embedder runs offline. See Network and egress. - Nothing between the data plane and the control plane ends TLS on the sync connection. In Istio clusters,
istio.enabledistrue, and the sync address’s port is inistio.excludeOutboundPorts(4317,4318,9090by default). - Clients reach the gateway through your ingress, with TLS ending there. See Expose the gateway to your clients.
- On a cluster with a NetworkPolicy controller,
embedder.networkPolicy.enabledrestricts the embedder to the gateway pods.
Availability
- At least two gateway replicas (
replicaCount). With the distributed store, the gateway runs as a StatefulSet sized bydistributed.replicaCount, which defaults to3. See Scale and run a highly available data plane. - If an autoscaler is on,
scalingConfig.scalingProperties.maxReplicasis above2. The chart’s autoscalers scale only the Deployment, so they don’t apply with the distributed store. - NTP runs on every Kubernetes node. Enrollment allows only 60 seconds of clock skew, and Gateway health flags a node whose clock is further off.
- The embedder doesn’t stall node drains: it runs two or more replicas, or has
embedder.podDisruptionBudget.maxUnavailable: 1. See Run the tool-search embedder. - The embedder’s model cache is on a volume (
embedder.persistence.enabled), so restarts don’t download the model again. With more than one embedder replica, or an autoscaler that can add one, setembedder.persistence.accessModesto[ReadWriteMany]or useembedder.persistence.existingClaim. With the defaultReadWriteOnce, keep one replica and setembedder.podDisruptionBudget.maxUnavailable: 1. - If more than one pod must share rate limits, a coordination Redis is configured, and
FABRIC_GW_RATE_LIMIT_COUNTER_STOREis"redis". It defaults toin_memory, which counts on each pod separately.
Secret store
With MySQL:
- The database is dedicated to the data plane, backed up with its encryption key, and reached over TLS, with
FABRIC_GW_MYSQL_TLSset to"true"underenv.vars. See Maintain the data plane and its data. - The migration user has DDL and data grants. If you want least privilege, the runtime user has only
SELECT,INSERT,UPDATE, andDELETE. See Use a least-privilege migration user.
With the distributed store:
-
FABRIC_GW_DISTRIBUTED_GOSSIP_KEYandFABRIC_GW_DISTRIBUTED_AUTH_PASSWORDare in the app Secret, under exactly these key names. Settingdistributed.gossipKeySecretKeyordistributed.authPasswordSecretKeyin your values has no effect. -
distributed.networkPolicy.enabledistrue, and your cluster enforces NetworkPolicies. The policy allows only the store’s ports,3320and3322, between gateway pods, so it also blocks client traffic to port8080. Allow it withdistributed.networkPolicy.extraIngress: -
Each pod has its own persistent volume.
Observability
- Telemetry reaches your backend, and you alert on
otelcol_exporter_send_failed_*if you use the collector sidecar. See Monitor the data plane. - You alert on pod restarts and on nodes that show Disconnected, Lagging, Degraded, or Critical in Gateway health. See What to alert on.
- You set
FABRIC_MCP_DEBUG_LOGSto"0", unless you want its extra diagnostic log lines for each MCP request. The chart turns it on.
Next steps
- To install with these settings, see Install the data plane with Helm.
- To keep the data plane current, see Upgrade the data plane.

