Privasys
Enclave OSEnclave OS (Virtual)

Architecture

The architecture of Enclave OS Virtual, running OCI containers inside Confidential VMs with end-to-end attestation via RA-TLS.

Enclave OS Virtual runs OCI containers inside Intel TDX (or AMD SEV-SNP) Confidential VMs with end-to-end attestation. Every container image digest, environment variable, volume mount, and platform configuration is measured into a deterministic Merkle tree and carried in X.509 certificate extensions via RA-TLS. Clients can verify the full workload stack on a single TLS connection: the certificate identifies the workload, the evidence exchange proves it.

High-Level Topology

┌────────────────────────────────────────────────────────┐
│  Intel TDX / AMD SEV-SNP Confidential VM               │
│                                                        │
│  ┌──────────────────────────────────────────────────┐  │
│  │  Enclave OS (Virtual)                            │  │
│  │                                                  │  │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────┐  │  │
│  │  │ ra-tls-    │  │ Workload   │  │ Management │  │  │
│  │  │ caddy      │  │ Launcher   │  │ Server     │  │  │
│  │  │ (TLS +     │  │ (containerd│  │ (HTTP on   │  │  │
│  │  │  reverse   │  │  lifecycle)│  │  localhost)│  │  │
│  │  │  proxy)    │  │            │  │            │  │  │
│  │  └─────┬──────┘  └─────┬──────┘  └────────────┘  │  │
│  │        │               │                         │  │
│  │        │    ┌──────────┴──────────┐              │  │
│  │        │    │     containerd      │              │  │
│  │        │    └──────────┬──────────┘              │  │
│  │        │    ┌──────────┴──────────┐              │  │
│  │        └────┤  OCI Containers     │              │  │
│  │             │  ┌─────┐  ┌──────┐  │              │  │
│  │             │  │App  │  │ DB   │  │              │  │
│  │             │  │     │  │      │  │              │  │
│  │             │  └─────┘  └──────┘  │              │  │
│  │             └─────────────────────┘              │  │
│  └──────────────────────────────────────────────────┘  │
│                                                        │
│  dm-verity root │ measured boot (RTMRs) │ erofs rootfs │
└────────────────────────────────────────────────────────┘

Three components run inside the Confidential VM:

  • Caddy RA-TLS is a Caddy server with a custom TLS issuance module that generates RA-TLS certificates. It acts as a reverse proxy, terminating TLS at the edge and forwarding plaintext to containers on localhost. Each container gets its own hostname and its own RA-TLS certificate with per-container OID extensions.
  • Workload Launcher manages the OCI container lifecycle via containerd. It pulls digest-pinned images, creates containers, runs health checks, and wires up Caddy routes dynamically.
  • Management Server exposes an HTTP API on localhost for loading and unloading containers at runtime, authenticated via OIDC bearer tokens. It also serves health, readiness, status, and Prometheus metrics endpoints over RA-TLS.

How It Works

  1. Boot. The VM starts from a dm-verity protected, fully measured image built with the Privasys cvm-images project. Every boot stage is measured into the TDX RTMRs, and every byte of the root filesystem is verified against a hash tree whose root is part of the measurement.

  2. Dynamic loading. The manager starts with zero containers. Operators call POST /api/v1/containers (authenticated via OIDC bearer token) to load containers at runtime. Each request specifies a digest-pinned OCI image reference, environment variables, volumes, and ports.

  3. Pull and verify. OCI images are pulled via containerd. Each image digest is verified against the pinned @sha256:... reference in the load request.

  4. Merkle tree. A platform Merkle tree and per-container Merkle trees are recomputed after every load or unload. Leaves include the CA certificate, image digests, environment variables, volume configuration, and runtime version.

  5. RA-TLS certificates. The Merkle roots and container metadata are embedded as X.509 extensions in RA-TLS certificates. The TDX quote is not in the certificate: it is served on the connection by the evidence exchange and bound to the certificate's key.

  6. TLS handshake and evidence. Clients connecting via TLS receive a certificate chain that states which OS image booted, which containers are deployed (by digest), and how they are configured; the evidence exchange then proves which TEE is running and that the certificate key lives inside it.

  7. Health and metrics. The management API exposes /healthz, /readyz, /api/v1/status, and Prometheus /metrics over RA-TLS at manager.<machine-name>.<hostname>.

  8. Trusted time. The manager never reads the guest's host-steered clock for a security decision. Token and voucher windows, quote freshness, certificate validity and peer verdicts read a trusted time, confirmed against the platform monitor's signed polls and NTS servers. Caddy's RA-TLS module takes its issuing time from the manager over a root-only local socket.

RA-TLS via Caddy

The RA-TLS module is a Caddy tls.issuance module that generates RA-TLS certificates inside the Confidential VM. It serves the same deterministic and challenge evidence exchange as Enclave OS Mini.

The module abstracts hardware-specific logic behind an Attester interface. Each TEE backend (TDX, SEV-SNP) lives in its own file and registers itself at init time.

Supported Backends

BackendHardwareStatus
tdxIntel TDXSupported
sev-snpAMD SEV-SNPPlanned

How Certificates Are Generated

Certificate (per hostname, 24 hours):

  1. An ECDSA P-256 key pair is generated inside the TEE.
  2. An X.509 certificate carrying the OID extensions is signed by the operator-provisioned intermediary CA. It carries no quote.
  3. The certificate is cached by certmagic and renewed every 24 hours; the key is kept across re-mints that change only extension values.

Evidence (on request, POST /__privasys/attest):

  1. The module checks that the request's leaf names a key it holds.
  2. Deterministic mode: the cached quote for that key is served, minted once with report_data = SHA-512( SHA-256(SPKI_DER) || quote_time ), where SPKI_DER is the DER-encoded SubjectPublicKeyInfo of the leaf public key and quote_time is the minting minute carried in the response.
  3. Challenge mode: hctx is derived from the connection's TLS exporter with the client's 32-byte context, and a fresh quote is obtained from the configured backend (/sys/kernel/config/tsm/report for TDX) with report_data = SHA-512( SHA-256(SPKI_DER) || context || hctx ).
  4. The response is served and the connection is tagged X-Privasys-Attestation: deterministic or challenge.

SNI-Based Routing

Each container gets its own hostname derived from the workload manifest:

  • Management API: manager.<machine-name>.<hostname>
  • Container myapp: myapp.<machine-name>.<hostname>
  • Internal containers (e.g. databases): no external hostname, accessible only on localhost

When a client connects, the RA-TLS module selects the certificate matching the SNI hostname. Each certificate carries per-container OID extensions (image digest, config Merkle root, volume encryption status).

Per-Container OID Extensions

Every container's RA-TLS certificate includes extensions specific to that container:

OIDNameValue
1.3.6.1.4.1.65230.4.1Container App IdPlatform-assigned app identity (apps.id, raw 16-byte UUID), stamped by the measured manager (omitted when the platform supplied none)
1.3.6.1.4.1.65230.4.2Container Image DigestRaw SHA-256 digest of the OCI image
1.3.6.1.4.1.65230.4.3Container Image RefFull OCI image reference string
1.3.6.1.4.1.65230.5.1Container Config Merkle RootSHA-256 root of the container's config tree
1.3.6.1.4.1.65230.6.1Container Volume Key Source"vault:<handle>" or "byok:<fingerprint>" (present only when an encrypted volume is attached)
1.3.6.1.4.1.65230.7.1Attested Dependency SetCanonical encoding of the container's pinned dependencies, stamped by the runtime

The platform-level certificate (served for the management hostname) carries the platform OIDs (arcs 1 to 3) including the runtime version hash (OID 1.1), the image profile (OID 1.2), the combined workloads hash (OID 2.4), and the data encryption key origin (OID 3.1).

Requirements

  • A Confidential VM running Intel TDX or AMD SEV-SNP
  • A hardened base image with dm-verity and measured boot
  • An intermediary CA certificate and private key (private PKI)
  • An OIDC provider for management API authentication
  • containerd installed on the VM
Edit on GitHub