Rust & Teaclave SGX SDK

Enclave OS is written entirely in Rust and built with a fork of the Apache Teaclave SGX SDK — the only mature Rust SDK for Intel SGX enclaves.

Why Rust for SGX

SGX enclaves impose unique constraints that make Rust an ideal choice:

Memory Safety Without a Runtime

SGX enclaves have no operating system inside the enclave boundary. There is no malloc allocator by default, no stack canaries from the OS, and no address-space layout randomization. A memory corruption bug inside an enclave can directly leak secrets.

Rust's ownership model eliminates entire classes of bugs — use-after-free, double-free, buffer overflows, data races — at compile time, without requiring a garbage collector or runtime.

No Standard Library (no_std)

Code inside an SGX enclave cannot use the full Rust standard library (std) because there is no OS to provide file I/O, networking, threads, or other syscalls. Enclave OS compiles in #![no_std] mode, using only:

• core — language primitives, traits, iterators.
• alloc — heap allocation via a custom allocator mapped to the SGX SDK's enclave allocator.
• SGX-specific crates for the remaining functionality.

Performance

Rust compiles to native code with zero-cost abstractions. There is no interpreter overhead, no JIT warmup, and no garbage collection pauses — critical inside an enclave where EPC memory is limited and context switches are expensive (~10,000 cycles per ECALL/OCALL).

The Teaclave SGX SDK

The Apache Teaclave SGX SDK (formerly Rust SGX SDK by Baidu) provides:

• SGX-compatible sysroot: A custom Rust standard library compiled against the SGX SDK's C runtime (sgx_tstdc), replacing POSIX syscalls with SGX equivalents.
• EDL bindings: Rust bindings for ECALLs and OCALLs defined in the Enclave Definition Language.
• SGX services: Wrappers for sgx_create_report, sgx_seal_data, sgx_read_rand, and other SGX primitives.
• Crypto: SGX-compatible versions of common crates (ring, rustls, webpki).

Why a Fork

Enclave OS uses a forked version of the Teaclave SDK rather than the upstream repository. The upstream SDK has not kept pace with Rust or Intel SGX SDK releases and does not build with modern toolchains. The fork maintains compatibility with the latest stable Rust and current Intel SGX SDK.

The key changes:

Some enclave-specific adaptations live in Enclave OS itself, not the Teaclave fork:

• getrandom — A vendored copy patched to route to sgx_read_rand (hardware RDRAND) via [patch.crates-io].
• rustls / webpki — Standard crate dependencies (rustls 0.23, rustls-webpki 0.102) configured with the ring crypto backend for SGX compatibility.

The Build System

Building an SGX enclave in Rust involves a multi-stage process:

1. Custom Sysroot (-Zbuild-std)

The enclave requires a custom Rust sysroot — core and alloc compiled from source against the SGX SDK's C library. The Teaclave fork provides a CMake sgx_sysroot target that builds it using Cargo's built-in -Zbuild-std=core,alloc:

# In the Teaclave fork's CMakeLists.txt
add_custom_target(sgx_sysroot
    COMMAND cargo +nightly-2025-12-01 build -Zbuild-std=core,alloc
            --target x86_64-unknown-linux-sgx.json
            --release
    ...
)

Enclave OS's top-level CMakeLists.txt invokes this target as an external project:

cmake -S ${TEACLAVE_CHECKOUT}
      -B ${SYSROOT_BUILD_DIR}
      -DBUILD_SYSROOT=ON
      -DSGX_SYSROOT_OUTPUT=${SGX_SYSROOT_DIR}
cmake --build ${SYSROOT_BUILD_DIR} --target sgx_sysroot

This produces core, alloc, and std rlibs that call into sgx_tstdc instead of glibc for all OS-level operations.

2. Enclave Compilation

The enclave Rust code is compiled with:

• Target: x86_64-unknown-linux-sgx — a custom target triple defined by the Teaclave SDK.
• Profile: release with size optimization (opt-level = "s", codegen-units = 1). LTO is disabled because the SGX sysroot rlibs lack LLVM bitcode sections.
• Flags: --sysroot <path> to use the custom sysroot, and -C target-feature=+rdrand to enable the hardware random number generator instruction.

3. Signing

The compiled enclave is signed with the SGX signing tool (sgx_sign) using:

• The enclave configuration (enclave.config.xml) specifying heap size, stack size, thread count, and debug mode.
• A 3072-bit RSA signing key that determines the MRSIGNER measurement.
• The resulting MRENCLAVE (SHA-256 of the enclave's initial state) is the primary identity used for attestation.

4. CMake Orchestration

The entire build is orchestrated by CMake, which:

1. Generates C headers from the EDL file via sgx_edger8r.
2. Compiles the host Rust code via Cargo.
3. Compiles the enclave Rust code with the custom sysroot (--sysroot pointing to the built rlibs).
4. Links the enclave binary against the SGX SDK's trusted runtime (sgx_trts, sgx_tservice, sgx_tcrypto).
5. Signs the enclave binary.
6. Copies the final artifacts to the build output directory.

Binary Size

Through aggressive optimization size-optimized profile (opt-level = "s"), single codegen unit, careful dependency selection, and symbol stripping the signed enclave binary is approximately 4 MB. This is remarkably small for a runtime that includes:

• A full TLS 1.3 stack (rustls)
• X.509 certificate generation (rcgen, yasna)
• AES-256-GCM encryption (ring)
• SHA-256/SHA-512 hashing
• WASM runtime (wasmtime)
• HTTP request routing
• SPSC queue communication