RPC & Circular Buffers

Enclave OS uses a custom RPC protocol over shared-memory circular buffers to communicate between the host (untrusted) and the enclave (trusted). This design minimizes the number of expensive ECALL/OCALL transitions while maintaining high throughput.

Why Not Standard ECALLs/OCALLs?

The SGX SDK provides ECALLs (host → enclave) and OCALLs (enclave → host) as the standard way to cross the enclave boundary. Each call involves:

• A context switch costing ~10,000 CPU cycles.
• Parameter marshalling — serializing and copying data across the boundary.
• EEXIT/EENTER instructions that flush the TLB and pipeline.

For a web server handling many requests per second, calling ECALLs/OCALLs for every read/write operation would be prohibitively slow.

Instead, Enclave OS makes one ECALL (ecall_run) to start the enclave's event loop, then all data flows through shared memory — memory regions that both the host and enclave can access. The only OCALL (ocall_notify) is a lightweight signal with no data payload.

SPSC Circular Buffers

The shared memory is organized as Single-Producer Single-Consumer (SPSC) circular buffers — a lock-free data structure where one thread writes and one thread reads, without any mutex or atomic compare-and-swap.

Memory Layout

┌──────────────────────────────────────────────────┐
│                   2 MB Buffer                    │
├──────────┬───────────────────────────┬───────────┤
│ Header   │         Data Region       │  Padding  │
│ (64B)    │                           │  (align)  │
└──────────┴───────────────────────────┴───────────┘

Header (64 bytes, cache-line aligned):
┌──────────┬──────────┬──────────────────────────┐
│ head: u64│ tail: u64│ padding (48 bytes)       │
│ (atomic) │ (atomic) │ (to fill cache line)     │
└──────────┴──────────┴──────────────────────────┘

Each connection gets two buffers: one for requests (host → enclave) and one for responses (enclave → host).

Key Design Decisions

Message Framing

Each message in the buffer is prefixed by a 4-byte header containing the message length (little-endian u32):

┌───────────┬──────────────────────┐
│ len: u32  │  payload (len bytes) │
│ (4 bytes) │                      │
└───────────┴──────────────────────┘

The constant MSG_HEADER_SIZE = 4 is defined in the shared common crate and used by both the host and enclave.

Write Path (Producer)

fn send(&self, data: &[u8]) -> Result<()> {
    let total = MSG_HEADER_SIZE + data.len();
    // 1. Read head and tail (atomic Acquire)
    // 2. Check available space (wrapping arithmetic)
    // 3. Write length header
    // 4. Write payload (may wrap around buffer end)
    // 5. Advance tail (atomic Release)
}

Read Path (Consumer)

fn recv(&self) -> Result<Vec<u8>> {
    // 1. Read head and tail (atomic Acquire)
    // 2. Check if data available
    // 3. Read length header
    // 4. Read payload (may wrap around buffer end)
    // 5. Advance head (atomic Release)
}

The wrapping logic handles the case where a message spans the end of the circular buffer — the write/read wraps around to the beginning.

RPC Protocol

On top of the raw circular buffers, Enclave OS implements a request-response RPC protocol. This provides multiplexing, method dispatch, and error handling.

Request Header (14 bytes)

┌──────────────┬───────────────┬─────────────────┐
│ req_id: u64  │ method: u16   │ payload_len: u32│
│ (8 bytes)    │ (2 bytes)     │ (4 bytes)       │
└──────────────┴───────────────┴─────────────────┘

Response Header (16 bytes)

┌──────────────┬───────────────┬─────────────────┐
│ req_id: u64  │ status: i32   │ payload_len: u32│
│ (8 bytes)    │ (4 bytes)     │ (4 bytes)       │
└──────────────┴───────────────┴─────────────────┘

The req_id field allows the enclave to match responses to requests, enabling potential future pipelining.

RPC Methods

The protocol defines 13 methods across four categories. Method IDs use hex ranges to group related operations:

Network Operations

Key-Value Store

Utility

Lifecycle

Serialization

Request and response payloads are serialized using a simple length-prefixed binary format (not protobuf, not JSON). This keeps the serialization code minimal and avoids pulling in large dependencies inside the enclave.

For example, a KvPut request payload:

┌──────────────┬───────────┬──────────────┬─────────────┐
│ key_len: u32 │ key bytes │ val_len: u32 │ value bytes │
└──────────────┴───────────┴──────────────┴─────────────┘

Performance Characteristics

The SPSC queue design means that the communication channel itself is essentially zero-overhead — the bottleneck is always the useful work (TLS handshake, HTTP parsing, WASM execution) rather than the data transport.