> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.fabricgateway.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.fabricgateway.ai/_mcp/server.

# Configure a secret store

> Choose the secret store and vault for a data plane you run, set up Azure Key Vault with managed identity or HashiCorp Vault, and fix common errors.

This page applies to the Enterprise plan, where you run the data plane in your own VPC. A data plane uses two kinds of secret storage. The **secret store** holds runtime state, such as the credentials your users connect. To choose one, see [Choose a secret store](/run/scale-and-high-availability#choose-a-secret-store). A **vault** holds the upstream secrets that you keep in your own system, such as an API key. This page explains how to choose each one, how to connect Azure Key Vault or HashiCorp Vault, and how to fix common errors.

## Prerequisites

Before you begin, make sure you have the following:

* A data plane that's connected to the control plane. See [Connect the data plane to the control plane](/run/connect-a-data-plane).
* The app Secret with an encryption key. See [Secrets and keys](/run/secrets-and-keys).
* For a vault, network access from the gateway pods to the vault.
* Access to the **Vaults** page under **Admin** in the console.

## Choose a secret store and a vault

| Layer        | Options                                                                   | Use it for                                                                                                                       |
| ------------ | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Secret store | `distributed-embedded`, `mysql`                                           | State that the data plane owns: credentials your users connect, OAuth state, and codes it issues. Always set.                    |
| Vault        | AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, HashiCorp Vault | Secrets that you store and rotate in your own system. Optional. Add one when a connection or plugin reads a secret by reference. |

Choose the secret store first, because the data plane needs it to start:

* **`distributed-embedded`** is the default on a data plane. The gateway pods keep the values in memory and replicate them to each other, so there's no database to run. It needs the gossip key and data-port password in the app Secret.
* **`mysql`** is optional. It stores the values in your MySQL database, for customers who prefer to run one. The Helm chart selects it unless you set `distributed.enabled: true`. When you use a vault for your secrets, you don't need it.

The `bbolt` store, which is a local file, isn't supported on a data plane. The gateway fails to start with `secrets.type=bbolt is not supported on the dataplane; use distributed-embedded (default) or mysql`.

Choose a vault by where your secrets already live. Use Azure Key Vault if you run on Azure, and HashiCorp Vault if you run it already. A vault is independent of the secret store, and you can register more than one.

## Settings

The data plane reads these environment variables for the secret store. In the Helm chart, set them under `env.vars`.

| Variable                                 | Default                                                                                                    | Required | What it does                                                                                                                                                                                                                       |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FABRIC_GW_SECRETS_TYPE`                 | `distributed-embedded` on a data plane. The chart sets `mysql` unless you set `distributed.enabled: true`. | No       | Selects the secret store: `distributed-embedded` or `mysql`.                                                                                                                                                                       |
| `FABRIC_GW_SECRETS_DSN`                  | Built from the `FABRIC_GW_MYSQL_*` variables                                                               | No       | The MySQL connection string, used only with `mysql`. Leave it unset to use the chart's MySQL settings.                                                                                                                             |
| `FABRIC_GW_SECRETS_VAULT_WRITE_FALLBACK` | `false`                                                                                                    | No       | When a write to your vault fails, such as when the identity lacks write permission, store the value in the data plane's encrypted store instead. The gateway logs the fallback. With the default, a failed write returns an error. |
| `FABRIC_GW_SECRET_KEY`                   | None                                                                                                       | Yes      | The key that encrypts stored values. See [Secrets and keys](/run/secrets-and-keys).                                                                                                                                                |

## Register a vault

You register a vault on the **Vaults** page. The control plane stores the vault's settings and sends them to the data plane. The data plane then reads and writes secrets in your vault itself, so secret values don't pass through the control plane.

1. In the console, select **Vaults** under **Admin**, and then select **Add**.
2. Enter a **Name**. You choose this name later, when you point a credential or plugin at the vault.
3. For the provider, select **Azure Key Vault** or **HashiCorp Vault**.
4. Enter the authentication fields in one of the following sections, and save the vault.

Both vault types have two optional settings:

| Setting     | Default | What it does                                                                                                                |
| ----------- | ------- | --------------------------------------------------------------------------------------------------------------------------- |
| `prefix`    | None    | The path prefix for the secrets that the gateway creates.                                                                   |
| `cache_ttl` | `60s`   | How long the gateway caches a value it read. Use a Go duration such as `30s` or `5m`. An invalid value falls back to `60s`. |

### Azure Key Vault

| Field           | Required                                              | What it does                                                                                                                                                                                                                                                 |
| --------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `vault_url`     | Yes                                                   | The vault's address, such as `https://myvault.vault.azure.net`. It must be HTTPS, with no port, path, or query. The host must be an Azure Key Vault host.                                                                                                    |
| `auth_mode`     | No                                                    | `managed_identity`, `workload_identity`, or `client_secret`. If you leave it empty, the gateway uses `client_secret` when you set a client secret, then `workload_identity` when the pod has `AZURE_FEDERATED_TOKEN_FILE`, and otherwise `managed_identity`. |
| `client_id`     | For `client_secret`. Optional for `managed_identity`. | With `managed_identity`, selects a user-assigned identity. Leave it empty to use the system-assigned identity.                                                                                                                                               |
| `tenant_id`     | For `client_secret`                                   | The Microsoft Entra tenant. Not valid with `managed_identity`, which uses the identity's own tenant.                                                                                                                                                         |
| `client_secret` | For `client_secret` only                              | The service principal's secret. Not valid with the other modes. The gateway stores it encrypted.                                                                                                                                                             |

The gateway doesn't use the developer credentials on a node, such as the Azure CLI. It uses only the mode you select.

Choose the mode that fits where the data plane runs:

* **`managed_identity`** for a node, VM scale set, or App Service identity. Set only `vault_url`, and add `client_id` for a user-assigned identity.
* **`workload_identity`** for AKS. The pod's ServiceAccount needs the `azure.workload.identity/client-id` annotation, and the pod needs the `azure.workload.identity/use=true` label. The AKS webhook then injects `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, and `AZURE_FEDERATED_TOKEN_FILE`.
* **`client_secret`** when neither is available. You manage the secret's rotation.

Give the identity one of these Azure roles on the vault:

| Role                      | Allows                                                                                      |
| ------------------------- | ------------------------------------------------------------------------------------------- |
| Key Vault Secrets User    | Reading secrets.                                                                            |
| Key Vault Secrets Officer | Reading, and also storing, rotating, and deleting the credentials that the gateway manages. |

Use a Key Vault that's dedicated to gateway-managed credentials. Key Vault deletes a whole secret, so if another writer adds versions to the same secrets, the gateway detects and rolls back the change, but it can't block it atomically.

Example for AKS with the Helm chart. The chart sets the ServiceAccount annotation through `serviceAccount.annotations`:

```yaml
serviceAccount:
  create: true
  annotations:
    azure.workload.identity/client-id: "<managed-identity-client-id>"
```

The chart has no value for pod labels, so it can't add the `azure.workload.identity/use=true` label for you. Add the label to the data plane's pod template yourself, for example with a Helm post-renderer or a Kustomize patch. If you can't, use `managed_identity` or `client_secret` instead.

Then register the vault in the console with `vault_url` set to `https://<vault-name>.vault.azure.net` and `auth_mode` set to `workload_identity`.

### HashiCorp Vault

| Field                     | Required         | What it does                                                                                                                                                                                      |
| ------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `address`                 | Yes              | The Vault address, such as `https://vault.internal:8200`. It must be an `http` or `https` URL with a host, and no path, query, or user. Private addresses are allowed.                            |
| `auth_type`               | Yes              | `token`, `approle`, or `kubernetes`.                                                                                                                                                              |
| `token`                   | For `token`      | A Vault token. The gateway stores it encrypted. It can't renew itself, so use a token that doesn't expire before you rotate it.                                                                   |
| `role_id` and `secret_id` | For `approle`    | The AppRole credentials. The gateway stores `secret_id` encrypted. Logs in on the `approle` auth mount.                                                                                           |
| `role`                    | For `kubernetes` | The Vault role that the pod's ServiceAccount token logs in as. The gateway reads the token from `/var/run/secrets/kubernetes.io/serviceaccount/token` and logs in on the `kubernetes` auth mount. |
| `namespace`               | No               | A Vault Enterprise namespace.                                                                                                                                                                     |
| `mount`                   | No               | The KV engine's mount path. Default: `secret`.                                                                                                                                                    |
| `kv_version`              | No               | `1` or `2`. Default: `2`.                                                                                                                                                                         |

With `approle` and `kubernetes`, the gateway logs in again before the token expires, and once more if Vault returns `403`.

Give the identity a Vault policy for the paths the gateway uses. For KV v2 with the default mount, the gateway reads `secret/data/<path>`. To store and delete secrets that it manages, it also writes `secret/data/<path>` and writes, reads, and deletes `secret/metadata/<path>`. It stamps each secret it creates with `managed-by: fabric-gateway`, and deletes only secrets that carry it. The health check calls `auth/token/lookup-self`.

```hcl
path "secret/data/*" {
  capabilities = ["read", "create", "update"]
}

path "secret/metadata/*" {
  capabilities = ["read", "create", "update", "delete"]
}
```

For read-only use, keep only `read` on `secret/data/*`. With KV v1, the gateway can read and write, but it declines to delete, because v1 can't carry the marker.

## Reference a secret

A credential or plugin points at a secret with a **vault** and a **secret reference**. In the console, select the vault by name, and then enter the reference. For a plugin, see [Secret Injector](/reference/plugins/secret-injector). Where you enter a secret directly, on a data plane that you run, you enter a reference instead of the value.

| Vault           | Reference you can enter                                                                                                               | Notes                                                                                                                                           |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| Azure Key Vault | The secret's name, or its full identifier: `https://<vault-name>.vault.azure.net/secrets/<name>` or `.../secrets/<name>/<version>`    | The identifier must belong to the vault you selected. Enter a **Version** only if you didn't put one in the identifier, and the two must match. |
| HashiCorp Vault | A path under the mount, such as `partner/api-key`, or the full read URL: `https://vault.internal:8200/v1/secret/data/partner/api-key` | The URL must match the vault's address and mount. A secret that the gateway stores is returned as the full URL. **Version** must be a number.   |

If a secret has more than one field, enter **Property** to pick one. A secret with a single `value` field needs no property.

For secrets that the gateway creates, such as a managed credential, the console stores and returns the full identifier or URL. Keep the full form for those. The gateway refuses a bare Azure name for a managed secret.

## Common errors

Each message is in the gateway log. The data plane redacts the secret reference.

| Message                                                                                                                                                                | Cause and fix                                                                                               |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `azure_key_vault: vault_url must be an https URL such as https://<name>.vault.azure.net`                                                                               | The URL isn't HTTPS. Correct it.                                                                            |
| `azure_key_vault: vault_url must not include a port, path, userinfo, query, or fragment`                                                                               | Use only `https://<name>.vault.azure.net`.                                                                  |
| `azure_key_vault: vault_url host ... is not an Azure Key Vault host`                                                                                                   | The host isn't a Key Vault domain. Check the name.                                                          |
| `azure_key_vault: unknown auth_mode ... (want managed_identity, workload_identity or client_secret)`                                                                   | Fix the `auth_mode` spelling.                                                                               |
| `azure_key_vault: client_secret auth requires auth.tenant_id and auth.client_id`, or `auth_mode client_secret requires auth.client_secret`                             | Add the missing fields, or switch to `managed_identity`.                                                    |
| `azure_key_vault: auth.client_secret must not be set with auth_mode ...` or `auth.tenant_id does not apply to auth_mode managed_identity`                              | Remove the field that the mode doesn't use.                                                                 |
| `azure credentials: workload_identity (is the pod labelled azure.workload.identity/use=true and its ServiceAccount annotated with azure.workload.identity/client-id?)` | Add the label and annotation, and restart the pods.                                                         |
| `azure kv get ...` or `azure kv set ...` with `403`                                                                                                                    | The identity lacks a role. Add Key Vault Secrets User to read, or Key Vault Secrets Officer to write.       |
| `azure kv: ref ... is not a secret in https://...`                                                                                                                     | The reference points at another vault. Use this vault's identifier or the secret's name.                    |
| `azure kv: secret exists and is not managed by fabric-gateway`                                                                                                         | The gateway won't overwrite or delete a secret it didn't create. Use a different name or a dedicated vault. |
| `azure kv: encoded secret name exceeds 127 characters; shorten the vault prefix`                                                                                       | Shorten the vault's `prefix`.                                                                               |
| `hashicorp_vault: unsupported auth_type ... (want token, approle, or kubernetes)`                                                                                      | Fix `auth_type`.                                                                                            |
| `hashicorp_vault: approle auth requires auth.role_id` (or `secret_id`, `role`, `token`)                                                                                | Add the missing field for that `auth_type`.                                                                 |
| `hashicorp_vault: kv_version ... must be 1 or 2`                                                                                                                       | Set `kv_version` to `1` or `2`, or leave it empty.                                                          |
| `hashicorp_vault: address ... must not include a path, query, fragment, or userinfo`                                                                                   | Use `scheme://host[:port]` only.                                                                            |
| `hashicorp_vault: reading kubernetes service account token`                                                                                                            | The pod has no ServiceAccount token at the expected path.                                                   |
| `hashicorp_vault ... login: ...`                                                                                                                                       | Vault rejected the login. Check the role, the auth mount, and the Vault policy.                             |
| `hashicorp_vault: ref ... is not a secret in ...`                                                                                                                      | The URL doesn't match the vault's address and mount. Use a path under the mount.                            |
| `hashicorp_vault: refusing redirect off the configured origin`                                                                                                         | Vault redirected to another host. Set `address` to the node that serves requests.                           |
| `hashicorp_vault delete skipped: secret is not managed by fabric-gateway` (warning)                                                                                    | The secret wasn't created by the gateway, so it stays.                                                      |

The console's **Dependencies** view shows whether your vault is reachable. See [Monitor the data plane](/run/monitor).

## Next steps

* To create the app Secret and rotate the encryption key, see [Secrets and keys](/run/secrets-and-keys).
* To store a vault secret as a plugin's credential, see [Secret Injector](/reference/plugins/secret-injector).
* For every chart setting, see [Helm values](/run/helm-values).
* To manage credentials and vaults in the console, see [Manage credentials and vaults](/configure/credentials).