Intel SGX

Strong Confidential Serverless Containers (SCSC) example

DISCLAIMER

Currently, Gateway to Queue Proxy or Gateway to Activator does not support TLS in Knative. The Knative dev's are currently still working on this. The problem is that Knative dynamically changes routes to put the queue proxy or activator behind the gateway. Currently only HTTP routes are used here with the HTTP header modified by Istio. Istio does not offer this option for TLS. Tried to change the routes to TLS, which resulted in Knative not working anymore, because on the one hand a network controller, the proxies and the Autoscaler need the metrics. Therefore, the Activator as well as the queue proxy cannot currently have a TLS endpoint. Only in the route Activato to Queue-Proxy this would be possible, but this alone does not make sense. This will be implemented as soon as the Knative dev's are finished.

Requirements

• Helm v3

• Kubernetes (Kubelet, Kubectl, Kubeadm)

• a container runtime

• Docker Installation for Ubuntu (for building and pushing the images)

  • Add current user to docker group, if you haven't already: sudo usermod -aG docker $USER and logout and login again from your OS. Now login to docker.

• Vault-CLI

• An azure VM capable of SGX with at least 4 vCPU's. Example was created on DC4s v3.

• install make and jq

You do not need a domain for this example, but of course you can use one! Read

FAQ for usage without a domain.

Preparation

1.1 Initialise a Kubernetes cluster and install a networking plugin of your choice. Here is an example using Calico (skip to 2.1 if you want to install a different networking plugin, but do 1.2 first)

1.2 Initialize your cluster. The pod-network-cidr is required by calico.

sudo kubeadm init --pod-network-cidr=192.168.0.0/16 

1.3 let the current user, use the kubeconfig:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Not recommended in production:

If you only have one node in your cluster, you need to use the control plane node to schedule pods on. Done by executing :

kubectl taint nodes --all  node-role.kubernetes.io/control-plane-

1.4

Clone the repository

git clone https://github.com/enclaive/morphism.git

Navigate to

cd ./morphism/deployments/azure_dcs

Set-up

This example uses the 'sed' command to replace some values. Meaning that, you have to pull the repository again, if you want to change any parameters you specified or manually change them in the appropiate files. Also you have to be in the directory of the Makefile at all times.

2.1 The Makefile expects your shell to be in the same folder, otherwise some commands won't work. Add the required helm repos with:

make add-helm-repos

2.2 Issue the command below for a quick-start. Read the FAQ before to understand the parameters.

make quick-start ip=your_ip load_balancer=true domain=example.com development=true pccs=false

Read the FAQ to understand, how to configure this command. Some Helm Charts take some time to install. Do not cancel ! If it takes over 2 Minutes, then something was configured wrong.

PCCS is not verified and is only for bare-metal. Azure has it's own certificate caching service. If pccs is set to true, you have to specify an additional parameter api_key=APIKEY. The api-key can be received from here by clicking on subscribe.

Replace {{ yourdomain.com }} in all commands below with your domain and remove the curly brackets.

2.3 First, set the vault address environment variable for the vault-CLI to know the vault's address.

export VAULT_ADDR="https://vault.{{ yourdomain.com }}"

Skip the TLS verification to receive the self-signed certificate of the Vault with the attestation report included. Verification of the attestation report is currently work-in-progress at Enclaive and will included later on. For now, the verification is skipped.

 export VAULT_SKIP_VERIFY="true"

curl https://vault.{{ yourdomain.com }}/premain/attest --insecure 

2.4 Initialize the vault and receive the unseal key + the admin token.

vault operator init -key-shares=1 -key-threshold=1

Note that, it is important to keep both values confidential in production and have to be stored securely. Unseal the vault by providing the unseal key.

vault operator unseal

Login to the vault by providing the token.

vault login

2.5 Almost done! Now the Vault needs to be configured. We are creating a Root CA and an Intermediate CA. The

make configure-vault domain={{ yourdomain.com }}

2.6 Lastly, deploy the example. For this to work, you have to push the image to a public image repository. A private one also works, but the Knative Service yaml files then have to include the imagePullSecret. Furthermore, this example assumes that the docker-cli is used, to push the image to the repository. To do this, you must be logged in to your docker account with the docker-cli.

The next command will build and push an image.

make actix image_name=username/image_name domain={{ yourdomain.com }}
make flask image_name=username/image_name domain={{ yourdomain.com }}

If you have a domain, you have to create a wildcard-DNS entry for *.default.{{your-domain.com}} or an entry for actix.default.{{your-domain.com}} and flask.default.{{your-domain.com}}. Then, you are able to see your secret message by calling:

 curl actix.default.{{ yourdomain.com }}/hello_world
 curl flask.default.{{ yourdomain.com }}/hello_world

That's it, have fun with Strong Serverless Confidential Containers

FAQ

How is the Vault configured?

The Vault creates a Root CA and an Intermediate CA. Only the intermediate CA signs certificates for the enclaves. If an enclave has a security vulnerability in its code that allows an attacker to read the private key of its certificate, the intermediate CA can be rotated. The actix and flask servers contact the Vault to get a certificate issued by the intermediate CA, which is used for TLS. This certificate is issued for their exact domain. By opening the Makefile, you can specify a SECRET that will be passed to the serverless container, here env will be saved as environment variables, args passed as arguments and files saved to the file system. A temporary file system is used here in Gramine, which resides in volatile memory and is therefore encrypted.