Entering and Exiting an Enclave

A new CPU mode is introduced for SGX, called enclave mode. The CPU enters enclave mode when it executes enclave code. Specifically, the CPU enters enclave mode with the EENTER or ERESUME instruction. While the CPU is in enclave mode, the CPU disables some debugging features such as hardware breakpoints.

The EENTER instruction can only be called by software running on ring 3, so the host process must be running on ring 3. This is to prevent the enclave from running with elevated privileges and potentially infecting the system it is running on, which is especially important for cloud providers. Note that system software is not allowed to execute the EENTER instruction and will result in an undefined instruction error. EENTER is passed an available TCS with at least one SSA as an argument. A TCS is said to be available if it is not currently being used by another logical processor. The SSA is needed to store the thread execution context in case of a hardware exception. First, the current instruction pointer (RIP) is set to the OENTRY in the TCS, then any registers that are modified are saved to restore them after the enclave execution. From the entry point, code execution can begin.

ERESUME is similar to EENTER and is called only after a hardware exception occurs. The difference is that in SSA, a thread context must be saved for recovery.

To exit an enclave, the EEXIT instruction is called and restores the registers saved by EENTER. It can also only be called when the processor is in enclave mode, and it switches the processor to non-enclave mode with the privilege level set to ring 3. Registers modified by enclave code are not automatically flushed on EEXIT and must be flushed by the enclave code to avoid information leakage.

Hardware exceptions are handled by an Asynchronous Enclave Exit (AEX). First, the current execution context is stored in an SSA within the TCS. Then, as in an EEXIT, the stored registers are restored. In addition, the return address in the register is modified to point to an asynchronous exit handler in the host process of the enclave. It then exits the enclave, switches the CPU mode to non-enclave mode, and runs the system software exception handler. Finally, the exception handler returns to the asynchronous exit handler, which then calls the ERESUME statement to re-enter the enclave.

SGX also supports page swapping to disk. This is not relevant to the understanding of this thesis and will not be covered here. Also, Kubernetes does not work with swapping enabled. I refer interested readers to 5.5 EPC page eviction in