Skip to content

Secrets management

Ampora reads several secrets at startup, stores some at rest (encrypted), and never logs any of them. This page is the canonical map of which is which, and how to rotate them.

Secrets read at startup

Secret Source Used for
ConnectionStrings:Ampora env var, K8s Secret Postgres connection
Authentication:Oidc:ClientSecret env var, K8s Secret OIDC client auth
KeyProtection:MasterKey env var, K8s Secret Wraps every encrypted-at-rest column
KeyProtection:PreviousMasterKey env var, K8s Secret Optional, only during rotation
Dispatch:RedisConnectionString env var, K8s Secret Only when Backplane=Redis and Redis requires auth
OpenTelemetry:Headers:Authorization env var, K8s Secret Optional, when the OTLP backend needs auth

Secrets persisted at rest (encrypted)

These are encrypted via IKeyProtector (AES-GCM-256 by default; HSM/KMS if configured) and stored in the database:

Secret Table Notes
CA private keys ca_signing_keys One row per Draft / Active / Trusted / Retired key
Bootstrap token plaintexts — Hashed (SHA-256), never stored in cleartext
Federation peer secrets (inbound + outbound) federation_peers Hashed; constant-time compared
GitOps source credentials (PAT or SSH key) git_config_sources AES-GCM-wrapped; no read-back
Token pool admin secrets token_pools Hashed
Tenant-scoped lint and policy bodies lint_rules, policy_definitions Cleartext (DSL source) but tenant-scoped

The encryption envelope on every column carries a small header: {scheme, key_id, nonce, tag}. Rotation rewraps the columns by re-reading each row, decrypting with the previous key, and writing back with the new — see Certificate rotation for the master-key rotation walkthrough.

Secret managers we have integration recipes for

Ampora itself does not depend on a specific secret manager — it reads strings at startup, however you delivered them. These recipes exist:

Manager Method Notes
External Secrets Operator ExternalSecret resources fronting AWS Secrets Manager / Azure Key Vault / GCP Secret Manager / Vault Recommended path on Kubernetes
Sealed Secrets Encrypted manifests checked into Git Easy if you already do GitOps for k8s
SOPS + Kustomize secretGenerator with SOPS-encrypted env files No runtime controller
HashiCorp Vault Agent Injector Mounted secret files via vault.hashicorp.com/agent-inject-secret-* Works well with the binary install too

For the binary install on a VM, the equivalent is reading the env file from a chmod-0640 file owned by root:ampora and populated by your config-management tool.

Rotation procedures

Postgres password

  1. Set the new password in Postgres (ALTER USER ampora WITH PASSWORD '...').
  2. Update the Secret.
  3. Rolling restart: kubectl rollout restart deploy/ampora-web.

There is a brief window where some pods have the old password and some have the new — both are valid in Postgres for a few seconds during a rolling update. Drop the old password from Postgres a few minutes after the rollout completes.

OIDC client secret

  1. Issue a new secret in the IdP. Do not delete the old one yet.
  2. Update the Secret.
  3. Rolling restart.
  4. After the rollout completes and login still works, delete the old secret in the IdP.

If you do this in the wrong order, login breaks while the rollout is in flight.

Master encryption key

The most delicate rotation; it touches every encrypted-at-rest column.

  1. Generate a new 32-byte CSPRNG key (openssl rand 32 | base64).
  2. Update the Secret:
  3. KeyProtection:MasterKey = new key.
  4. KeyProtection:PreviousMasterKey = current key.
  5. Rolling restart. Ampora now reads with whichever key matches the stored envelope.
  6. Settings → PKI → Re-wrap encrypted columns. This walks every secret column and re-wraps it under the new key. Time scales with number of agents and configs; usually under a minute.
  7. Confirm via the page's "Last rewrap UTC" timestamp.
  8. Update the Secret again, this time removing KeyProtection:PreviousMasterKey.
  9. Rolling restart. The previous key is now retired.

If a step fails between 4 and 6, you can re-run the re-wrap idempotently — it skips columns already wrapped under the new key.

Federation peer secret

  1. On either side: Settings → Federation → {peer} → Rotate inbound secret (or outbound). Copy the new plaintext.
  2. On the other side: paste it into the matching column.
  3. Confirm with Ping.

If you do this in the wrong order or forget to update one side, calls will start returning 401 — the audit log on both sides shows which factor failed.

GitOps source credential

Credentials cannot be edited in place. Settings → GitOps → {source} → Delete, then re-create with the rotated PAT or SSH key. Existing imported configurations are kept; the next sweep continues from the new credential.

What never gets logged

  • Plaintext bootstrap tokens. They are emitted to the operator's browser once, never to logs.
  • Decrypted at-rest secrets. The decryption sites are exclusively in-memory; logs only ever see the wrapped form.
  • Connection strings with passwords. The Postgres password is redacted in any log line that mentions the connection string.
  • OIDC client secret. Only the token endpoint exchange uses it; that call is not logged.

If you discover a log line that contains any of these, it is a security bug — please report via the disclosure channel in SECURITY.md.