Skip to content

Runtime Cloud Federation

Runtime Cloud is for cross-site dispatch, identity, and policy. Local mesh/discovery is the smaller path inside one trusted network boundary.

Use local discovery or mesh first when the deployment stays inside one trusted network boundary.

Quickstart

Goal: bring two runtimes online, verify discovery/state projection, and perform cross-runtime dispatch with deterministic preflight.

1) Prerequisites

  • trust-runtime available in PATH
  • Two terminals
  • curl and jq
  • Linux/macOS shell

2) Bootstrap Two Runtime Projects

Create project folders once (first run creates bundle layout):

mkdir -p ~/trust-cloud-demo
trust-runtime play --project ~/trust-cloud-demo/runtime-a
# Stop with Ctrl+C after startup messages appear.
trust-runtime play --project ~/trust-cloud-demo/runtime-b
# Stop with Ctrl+C after startup messages appear.

3) Apply Known-Good Dev Profile Configs

cp examples/runtime_cloud/runtime-a-dev.toml ~/trust-cloud-demo/runtime-a/runtime.toml
cp examples/runtime_cloud/runtime-b-dev.toml ~/trust-cloud-demo/runtime-b/runtime.toml

If ports 18081, 18082, 5201, or 5202 are already in use on your host, update listen values in the copied runtime.toml files before startup.

The example configs include explicit mesh baseline keys:

  • runtime.mesh.role = "peer"
  • runtime.mesh.connect = []
  • runtime.mesh.zenohd_version = "1.7.2"

4) Start Both Runtimes

Terminal 1:

trust-runtime play --project ~/trust-cloud-demo/runtime-a

Terminal 2:

trust-runtime play --project ~/trust-cloud-demo/runtime-b

5) Verify Runtime Cloud State

curl -s http://127.0.0.1:18081/api/runtime-cloud/state | jq '{context: .context, nodes: .topology.nodes, edges: .topology.edges}'

Expected:

  • context.connected_via exists
  • topology.nodes contains runtime-a and runtime-b
  • topology.edges shows communication links

6) Preflight a Cross-Runtime Read

curl -s http://127.0.0.1:18081/api/runtime-cloud/actions/preflight \
  -H 'content-type: application/json' \
  -d @examples/runtime_cloud/dispatch-status-read.json | jq

Expected:

  • allowed: true
  • no denial_code
curl -s http://127.0.0.1:18081/api/runtime-cloud/actions/dispatch \
  -H 'content-type: application/json' \
  -d @examples/runtime_cloud/dispatch-status-read.json | jq

Expected:

  • top-level ok: true
  • results[0].runtime_id == "runtime-b"
  • results[0].audit_id present

8) Optional: Local Config Apply Through Runtime Cloud Dispatch

curl -s http://127.0.0.1:18081/api/runtime-cloud/actions/dispatch \
  -H 'content-type: application/json' \
  -d '{
    "api_version": "1.0",
    "request_id": "req-quickstart-cfg-1",
    "connected_via": "runtime-a",
    "target_runtimes": ["runtime-a"],
    "actor": "spiffe://trust/default-site/engineer-1",
    "action_type": "cfg_apply",
    "dry_run": false,
    "payload": { "params": { "log.level": "debug" } }
  }' | jq

Expected:

  • ok: true
  • per-target audit_id present

9) Next Steps

  • docs/guides/RUNTIME_CLOUD_PROFILE_COOKBOOK.md
  • docs/guides/RUNTIME_CLOUD_FEDERATION_GUIDE.md
  • docs/guides/RUNTIME_CLOUD_UI_WALKTHROUGH.md

Federation Guide

Goal: enable cross-site runtime cloud control safely with explicit allowlists and auditable outcomes.

Important:

  • The TOML blocks in this document are policy overlays, not complete runtime.toml files.
  • Start from a generated baseline (trust-runtime wizard --path <project>), then merge these sections.
  • For complete runnable examples, use examples/runtime_cloud/runtime-*.toml.

1) Federation Model

  • A local site can act on remote-site runtimes through runtime cloud dispatch
  • Cross-site write actions are denied by default in wan unless explicitly allowlisted
  • Dry-run preflight is mandatory before cross-site writes
  • Mesh/version baseline must stay aligned to Zenoh 1.7.2 across runtimes/gateways unless an approved exception exists

2) Runtime ID and Actor Conventions

Use consistent identities:

  • Runtime IDs: <site>/<runtime> (example: site-b/runtime-b)
  • Actor identities: spiffe://trust/<site>/<principal>

3) Baseline WAN Configuration

On the egress runtime (originating cross-site actions):

# Overlay only
[runtime.control]
auth_token = "REPLACE_WITH_LONG_RANDOM_TOKEN"

[runtime.web]
auth = "token"
tls = true

[runtime.tls]
mode = "self-managed"
cert_path = "./certs/runtime.crt"
key_path = "./certs/runtime.key"

[runtime.cloud]
profile = "wan"

[runtime.cloud.wan]
allow_write = []

4) Verify Default Cross-Site Deny

curl -s http://127.0.0.1:18081/api/runtime-cloud/actions/preflight \
  -H 'content-type: application/json' \
  -d @examples/runtime_cloud/preflight-cross-site-deny.json | jq

Expected:

  • allowed: false
  • denial_code: "permission_denied"
  • reason references cross-site write + explicit allowlist requirement

5) Add Explicit Allowlist Rule

# Overlay only
[runtime.cloud.wan]
allow_write = [
  { action = "cfg_apply", target = "site-b/*" }
]

Re-run preflight:

curl -s http://127.0.0.1:18081/api/runtime-cloud/actions/preflight \
  -H 'content-type: application/json' \
  -d @examples/runtime_cloud/preflight-cross-site-allow.json | jq

Expected:

  • allowed: true
  • no denial_code

6) Dispatch and Audit Correlation

After preflight success, dispatch to /api/runtime-cloud/actions/dispatch.

Required checks:

  • top-level request_id matches caller-generated ID
  • each target result contains audit_id
  • audit stream contains corresponding event with same request correlation

7) Security Guardrails

  • Avoid wildcard * targets in production unless formally approved
  • Keep action-specific rules (cfg_apply and cmd_invoke) separate
  • Prefer exact target IDs for high-impact commands
  • Revoke by setting allow_write = [] and reloading config

8) Deterministic Failure Modes

  • Missing WAN allowlist match -> permission_denied
  • Target without secure transport metadata in plant/wan -> deterministic deny
  • Actor role too low -> ACL denial code (acl_denied_cfg_write for config writes)

9) Evidence Requirements

For each federation policy change:

  • capture preflight deny/allow output
  • capture dispatch output with request_id and audit_id
  • archive logs under your gate artifact path