Skip to content

Federate two clusters

You will federate two Ampora servers — call them Ampora-EU and Ampora-US — so an operator on either side can see the other side's fleet read-only. Federation is the lightweight building block beneath "single pane of glass for multi-region operators."

Time: ~ 15 minutes (per side).

Prerequisites

  • Two Ampora servers, each reachable from the other over HTTPS (mTLS).
  • Admin login on both.
  • Each server has a stable hostname its peer will connect to. For this tutorial: ampora-eu.example.com and ampora-us.example.com.

What federation is and is not

Is: a read-only aggregation surface. EU operators can list and inspect US agents and vice versa. The aggregated Fleet view shows a clear "Includes federation" badge and namespaces remote agents (peer:us-region/<id>).

Is not: a global control plane. There is no shared identity provider, no shared tenant ID, no automatic write-coordination. Federated rollouts and cross-cluster handover ride on top in ADR-051 but require explicit operator intent.

1. Generate a federation client cert (each side)

Both servers act as mTLS clients when calling each other. Federation auth is mTLS plus a shared secret — both must match.

On Ampora-EU:

  1. Settings → Federation → Issue client cert.
  2. Subject: ampora-eu (free text, used in audit logs).
  3. Validity: 90d.
  4. Issue.

A modal shows:

  • the public certificate PEM (copy this, you need it on the US side),
  • the private key stays on this server only.

Repeat on Ampora-US to generate ampora-us's public cert.

2. Generate the shared secrets (each side)

Each peer entry holds two secrets: one used to authenticate the remote server when it talks to us (inbound), and one used by us when we talk to the remote (outbound). Both are 32-byte CSPRNG tokens.

On Ampora-EU:

  1. Settings → Federation → New peer.
  2. Name: us-region.
  3. Endpoint URL: https://ampora-us.example.com.
  4. Remote public cert PEM: paste what Ampora-US gave you in step 1.
  5. Click Generate inbound secret — copy the plaintext (shown once).
  6. Click Generate outbound secret — copy the plaintext (shown once).
  7. Save.

The row stores the hashes of both secrets. The plaintexts you copied are the values the other side needs in their corresponding peer row.

Repeat on Ampora-US:

  1. New peer, name eu-region, endpoint https://ampora-eu.example.com.
  2. Paste Ampora-EU's public cert.
  3. Paste Ampora-EU's outbound secret as inbound here.
  4. Paste Ampora-EU's inbound secret as outbound here.
  5. Save.

The "swap inbound and outbound between sides" rule sounds odd at first — the secret we use outbound is what the other side checks inbound.

3. Verify connectivity

On Ampora-EU → Settings → Federation → us-region, click Ping. The UI calls GET /api/federation/ping on Ampora-US with both factors. A green "200 OK · 47 ms" response means:

  • mTLS handshake succeeded (cert thumbprint matches the pinned value),
  • the outbound shared secret matched on the receiving side.

A red error means one of:

  • TLS handshake failed (wrong cert PEM, hostname mismatch, expired cert) — see the inline TLS error string,
  • 401 (the shared secret did not match) — re-paste both,
  • 5xx (the remote server's federation auth handler is unhappy) — the remote audit log will tell you which check failed.

4. See the remote fleet

Open Fleet → Federation toggle. A new section shows agents from each peer, namespaced like peer:us-region/<id>.

You can:

  • click an agent to open its details (read-only — buttons that would mutate state on the remote side are disabled),
  • filter by peer,
  • search across peers — filtering server-side, fan-out happens via IHttpClientFactory clients with the per-peer client cert.

If a peer is unreachable, its rows are omitted but the card has an Unreachable badge and the timestamp of the last successful contact — partial results never fail the whole list.

5. Audit federation use

Two audit streams are relevant:

  • On the calling side: an Federation.PeerCalled event for every outbound aggregator request. High-volume — keep an eye on retention if you list often.
  • On the receiving side: a Federation.RemoteRequest event with the remote peer name and the endpoint hit.

Both are also exported as Ampora self-observability metrics (ampora_federation_calls_total{peer,direction,outcome}) so you can graph them in your Prometheus.

6. Set up cross-cluster handover (optional)

Cross-cluster handover lets a single agent move from one Ampora server to another while keeping its identity continuity. It is a Tranche-4 feature on top of federation and uses a signed handover token. Follow the Federation → Handover guide if you need it; it is not required for the read-only aggregation flow above.

What you learned

  • That federation is bilateral, manual, and authenticated by mTLS + a shared secret.
  • How to issue federation client certs and rotate-friendly shared secrets.
  • How the aggregated Fleet view namespaces remote agents.
  • How to verify connectivity and audit federation use.

Next