Privasys
Enclave OSEnclave OS (Virtual)

Hardened Guest Images

The dm-verity base image for Enclave OS Virtual, the trust chain from silicon to userland, and why minimal images matter for confidential computing.

Enclave OS Virtual boots from a minimal, read-only, fully measured VM image built with mkosi. Every byte of the root filesystem is verified by dm-verity against a hash tree whose root hash is measured by TDX into the RTMR registers. A remote verifier can check the RTMR values and know, cryptographically, that the VM is running exactly the code that was built from the tdx-image-base repository.

Trust Chain

Silicon (TDX hardware)
  └─ MRTD     measures the TD firmware (OVMF/TDVF) loaded by the hypervisor
      └─ RTMR[0]     measures the firmware configuration (CC MR 1)
          └─ Secure Boot     UEFI verifies shimx64.efi → grubx64.efi → kernel
              └─ RTMR[1]     measures the EFI boot path: shim and GRUB binaries (CC MR 2)
                  └─ RTMR[2]     measures OS boot: kernel, initrd, cmdline with dm-verity root hash (CC MR 3)
                      └─ dm-verity     every block of the rootfs verified against the hash tree
                          └─ All userland binaries     any modification = I/O error + kernel panic

Every byte of code that executes on the machine is either measured by TDX hardware or verified by dm-verity. There are no gaps in the chain.

Why a Custom Image

General-purpose Confidential VM images (such as Google's pre-built Ubuntu images) are designed to run any workload. That generality is at odds with verifiability. A confidential VM is only as trustworthy as the code running inside it.

General-purpose CVM imagetdx-image-base
Root filesystemext4 (read-write)erofs (read-only)
dm-verityNot enabledEnabled on every block
Installed packages~2000+ (full Ubuntu stack, cloud agents, package managers)~40 (minimal: kernel, systemd, openssh, attestation tools)
Image size~30 GB~1.5 GB
Can modify rootfs at runtimeYesNo (I/O error then kernel panic)
Kernel modulesAll modules including unsigned third-party driversSigned modules only (module.sig_enforce=1)
Kernel lockdownNot enforcedlockdown=integrity (no unsigned code in ring 0)
Attack surfaceLarge: writable FS, package managers, update servicesMinimal: read-only FS, no package manager, no writable paths except tmpfs and data partition
What TDX attests"Some image the cloud provider built""This exact erofs image with this exact dm-verity root hash, bit-for-bit"

The core problem with general-purpose images:

  1. The rootfs is writable, so software can be installed or replaced after boot. The TDX measurement covers the initial state, but the running state can drift.
  2. Without dm-verity, nothing prevents a compromised process from modifying binaries on disk.
  3. Thousands of packages means a large attack surface.
  4. Unsigned kernel modules can be loaded, giving arbitrary code full ring-0 access to TEE-protected memory.

With tdx-image-base, the dm-verity root hash is baked into the kernel command line and measured by TDX. A remote verifier can check the RTMR values against the expected hash and confirm the VM is running exactly the code in the repository.

What Is in the Image

ComponentDetails
Guest OSUbuntu 24.04 LTS (Noble Numbat)
Kernellinux-image-gcp (6.17+, with TDX guest support)
Root filesystemerofs (read-only)
Integritydm-verity hash tree
BootSigned shim (Microsoft) then signed GRUB (Canonical) then kernel + initrd + dm-verity root hash in cmdline
Secure BootEnabled: full chain from UEFI firmware through bootloader to kernel
PartitionsESP (512 MB) + root erofs (~940 MB) + verity hash (~63 MB)
Networkingsystemd-networkd with DHCP
SSHopenssh-server (password auth disabled)
Attestation supporttpm2-tools, clevis, cryptsetup
Cloud integrationgoogle-compute-engine, google-guest-agent (removable for other platforms)

erofs and dm-verity

erofs (Enhanced Read-Only File System) is a read-only filesystem designed for small size and fast random reads. It is ideal for dm-verity because writes are impossible by design, not just by policy. Any attempt to write to the root filesystem results in an immediate I/O error.

dm-verity is a kernel device-mapper target that provides transparent integrity checking. Every 4 KB block of the root partition has a corresponding SHA-256 hash in the verity hash partition. When a block is read, the kernel computes its hash and compares it against the stored value. If the hashes do not match, the read fails and the kernel panics.

Together, they guarantee that the root filesystem is bit-for-bit identical to what was built. No runtime modification is possible.

Where It Fits in the TDX Stack

┌─────────────────────────────────────────────────────────────┐
│  TDX SEAM module                          Silicon firmware  │
│  Creates and isolates Trust Domains, manages memory keys    │
└──────────────────────────┬──────────────────────────────────┘
                           │ creates / measures
┌──────────────────────────▼──────────────────────────────────┐
│  Host OS / hypervisor                                       │
│  Modified kernel, QEMU, OVMF/TDVF                           │
│  Enables the host to launch TDX guests                      │
└──────────────────────────┬──────────────────────────────────┘
                           │ launches
┌──────────────────────────▼──────────────────────────────────┐
│  tdx-image-base                             Guest OS image  │
│  GRUB boot, erofs root, dm-verity, attestation tools        │
└──────────────────────────┬──────────────────────────────────┘
                           │ services go here
┌──────────────────────────▼──────────────────────────────────┐
│  Enclave OS Virtual                                         │
│  ra-tls-caddy, containerd, management server, containers    │
└─────────────────────────────────────────────────────────────┘

The guest OS image is the foundation. Application-specific layers (ra-tls-caddy, containerd, the management server) are built on top of it.

Releases and Measurements

Each tagged release of the base image publishes:

ArtifactDescription
dm-verity root hashThe SHA-256 root of the hash tree covering every byte on the read-only root filesystem. This is the primary code identity measurement.
Disk imageBootable TDX Confidential VM image (cloud-specific formats produced by CI)

The dm-verity root hash is embedded in the kernel command line (roothash=...) and measured into RTMR[2] at boot (via CC MR 3). RTMR[1] (CC MR 2) measures the boot manager code (shim and GRUB binaries). Together, a remote verifier can confirm the exact bootloader AND root filesystem by checking RTMR[1] and RTMR[2] in the TDX quote.

RTMR Mapping

On Google Cloud's TDX implementation, the CC Event Log (CCEL) uses CC Measurement Register indices that map to TDX RTMRs:

CC MRRTMRTCG PCRsWhat it measures
1RTMR[0]PCR 0Firmware code (TDVF/OVMF)
2RTMR[1]PCR 1–7EFI boot path: shim binary, GRUB binary, Secure Boot variables, EFI actions
3RTMR[2]PCR 8–15OS boot: GRUB commands, kernel binary, initrd, kernel command line (including roothash=), MOK list
4RTMR[3]PCR 16+Application-defined (unused)

The dm-verity root hash appears in the kernel command line, which GRUB measures into CC MR 3 (RTMR[2]). RTMR[1] captures the bootloader binaries themselves, it changes when the shim or GRUB is updated, but not when the root filesystem changes.

Verifying RTMRs from the Event Log

You can independently verify that the RTMR values in a TDX quote match the CC Event Log by replaying the SHA-384 extend operations. Each RTMR starts at all zeros and is extended by each measurement event:

RTMR[i] = SHA-384( RTMR[i] ‖ event_digest )

Reading the event log

The CCEL event log is available inside the VM at /sys/firmware/acpi/tables/data/CCEL. Parse it and replay the extends:

# Install tpm2-tools (if not already present)
sudo apt-get install -y tpm2-tools

# Dump the event log in human-readable format
sudo tpm2_eventlog /sys/kernel/security/tpm0/binary_bios_measurements

Finding the dm-verity root hash

Search the event log for the kernel command line event. On a tdx-image-base image, it contains the dm-verity root hash:

sudo tpm2_eventlog /sys/kernel/security/tpm0/binary_bios_measurements \
  | grep -A2 "kernel_cmdline"
# Example output:
#   kernel_cmdline: /vmlinuz-6.17.0-1009-gcp root=PARTUUID=... roothash=b93b4ee5...

Replaying RTMR extends (Go)

The following Go program parses the CCEL event log, replays the SHA-384 extends, and prints the computed RTMR values. Compare the output with the RTMR values from the TDX attestation quote.

package main

import (
	"crypto/sha512"
	"encoding/binary"
	"encoding/hex"
	"fmt"
	"io"
	"os"
)

const (
	algSHA384  = 0x000c
	sha384Size = 48
	evNoAction = 0x00000003
)

// CC MR index → RTMR mapping (CCEL format)
func ccmrToRTMR(ccmr uint32) int {
	switch ccmr {
	case 1:
		return 0 // RTMR[0]
	case 2:
		return 1 // RTMR[1]
	case 3:
		return 2 // RTMR[2]
	case 4:
		return 3 // RTMR[3]
	default:
		return -1
	}
}

func main() {
	path := "/sys/firmware/acpi/tables/data/CCEL"
	if len(os.Args) > 1 {
		path = os.Args[1]
	}

	f, err := os.Open(path)
	if err != nil {
		fmt.Fprintf(os.Stderr, "open: %v\n", err)
		os.Exit(1)
	}
	defer f.Close()

	// Skip TCG_EfiSpecIDEventStruct header (legacy format)
	var pcr0 uint32
	var evType0 uint32
	var digest0 [20]byte
	var evSize0 uint32
	binary.Read(f, binary.LittleEndian, &pcr0)
	binary.Read(f, binary.LittleEndian, &evType0)
	io.ReadFull(f, digest0[:])
	binary.Read(f, binary.LittleEndian, &evSize0)
	headerData := make([]byte, evSize0)
	io.ReadFull(f, headerData)

	// Parse algorithm descriptors from SpecID header
	numAlgs := binary.LittleEndian.Uint32(headerData[24:28])
	type algDesc struct{ id, size uint16 }
	algs := make([]algDesc, numAlgs)
	for i, off := uint32(0), 28; i < numAlgs; i, off = i+1, off+4 {
		algs[i] = algDesc{
			binary.LittleEndian.Uint16(headerData[off:]),
			binary.LittleEndian.Uint16(headerData[off+2:]),
		}
	}

	// Replay extends
	var rtmr [4][sha384Size]byte

	for evNum := 1; ; evNum++ {
		var ccmr, eventType, digestCount uint32
		if binary.Read(f, binary.LittleEndian, &ccmr) != nil {
			break
		}
		binary.Read(f, binary.LittleEndian, &eventType)
		binary.Read(f, binary.LittleEndian, &digestCount)

		var sha384Digest [sha384Size]byte
		for i := uint32(0); i < digestCount; i++ {
			var algID uint16
			binary.Read(f, binary.LittleEndian, &algID)
			if algID == 0xffff {
				goto done
			}
			var size uint16
			for _, a := range algs {
				if a.id == algID {
					size = a.size
					break
				}
			}
			digest := make([]byte, size)
			io.ReadFull(f, digest)
			if algID == algSHA384 {
				copy(sha384Digest[:], digest)
			}
		}

		var evDataSize uint32
		binary.Read(f, binary.LittleEndian, &evDataSize)
		evData := make([]byte, evDataSize)
		io.ReadFull(f, evData)

		if eventType == evNoAction {
			continue
		}
		idx := ccmrToRTMR(ccmr)
		if idx < 0 || idx > 3 {
			continue
		}
		h := sha512.New384()
		h.Write(rtmr[idx][:])
		h.Write(sha384Digest[:])
		copy(rtmr[idx][:], h.Sum(nil))
	}
done:
	for i := 0; i < 4; i++ {
		fmt.Printf("RTMR[%d] = %s\n", i, hex.EncodeToString(rtmr[i][:]))
	}
}

Run it on the VM and compare with the attestation quote:

go build -o rtmr-replay . && sudo ./rtmr-replay
# RTMR[0] = 75d9f1d1...
# RTMR[1] = 9cc6d43f...
# RTMR[2] = c8a9e472...   ← contains the dm-verity root hash measurement
# RTMR[3] = 000000000...

If the computed values match the RTMR values in the TDX attestation quote, the event log is authentic and the boot chain is verified.

How Updates Work

The rootfs is read-only. Running apt install on a booted VM is impossible. To update:

  1. Edit the configuration in the tdx-image-base repository (add or update packages, bump the image version).
  2. Build the image with sudo mkosi build.
  3. Test locally with QEMU.
  4. Upload and register the new image on the target cloud platform.
  5. Create a new VM, attach the existing data disk, and delete the old VM.

The data partition (LUKS-encrypted, separate persistent disk) survives image updates.

Design Choices

DecisionRationale
erofs over ext4Read-only by design. Smaller than ext4, ideal for dm-verity. No accidental writes possible.
GRUB over systemd-bootGCP's TDX firmware (TDVF) enforces Secure Boot which silently rejects unsigned EFI binaries. GRUB is the proven boot chain for TDX across cloud platforms.
Minimal package setEvery additional package increases the attack surface. ~40 packages means fewer potential CVEs than ~2000.
Signed kernel modules onlymodule.sig_enforce=1 plus lockdown=integrity prevents unsigned code from running in ring 0.
Edit on GitHub