Knative
Last updated
Last updated
Originally developed by Google, Knative is now an open source project developed by more than 50 different companies. The goal of Knative is to bring the benefits of serverless computing, more specifically serverless containers, to a Kubernetes cluster. Auto-scaling, load balancing, and traffic management of serverless applications is done automatically by Knative. Knative runs on top of Kubernetes, which is achieved by defining custom resource definitions (CRDs) with three components: Build, Serving, and Eventing. In this thesis, we will focus primarily on Knative's Serving component.
Serving is used for serverless containerized web applications. For each serverless container, a Knative service is deployed, which consists of three additional resources. The Knative Service manages the lifecycle of the container and deploys the additional resources.
Configurations stores the Knative and Kubernetes deployment configuration for a Knative Service. For Knative, this includes, for example, how scalement is done. It also stores available Kubernetes pod configuration options for serverless containers, such as volume mounts, images, ports, and environment variables.
Revisions are container snapshots at a given point in time and created upon configuration updates and code updates. They are especially useful in production to facilitate the update process without downtime and to test new updates. Once a revision is no longer needed, it can be deleted by an administrator. Older revisions are automatically deleted after a specified period of time.
Routes are network endpoints for revisions and are used for traffic management. Traffic can be split between revisions, which is expressed in percentage. This makes it possible to deploy the update first and then gradually route all traffic to the new version, or to partially route some users to the new update to test it for bugs, etc. Knative must be configured with a domain in production. Each route created by a service internally creates a subdomain and configures the gateway to route incoming requests for the created subdomain to the Knative service. In order for this to work efficiently, a DNS wildcard entry must be created for Knatives subdomain range.
Knative requires a networking plug-in to route all traffic between and to its components. In addition, Knative introduces several architectural components to achieve horizontal autoscaling: the activator, the autoscaler, and the queue-proxy, see the figure above. The autoscaler actually does the scaling by collecting metrics from the queue-proxy and the activator. For example, the queue-proxy and the activator send the number of concurrent requests to the autoscaler, which then decides whether to scale the application. Scaling is done by calling the Kubernetes API to configure the pod replica set. In Knative, it is possible to scale the application to zero, meaning that there is currently no user application deployed to handle any requests. To still serve incoming traffic, the activator is introduced. Deploying an application that has been scaled to zero is called a cold start. The Activator is mainly used during a cold start and when the amount of traffic is low. IIts purpose is mainly to act as a buffer on cold starts for HTTP requests reaching the user application. It also informs the autoscaler about traffic metrics, indirectly telling the autoscaler to start the application. When the application is successfully deployed, the Activator sends all buffered requests to the Kubernetes Service, which then forwards them to the user application's queue-proxy.
If the autoscaler receives metrics of high incoming traffic and the application is already up, the autoscaler will edit the istio VirtualService to route all traffic destined for the user application directly from the gateway to the application without having the activator in its path, and put it back in the path when no or low traffic is received. Since the istio VirtualService is a Kubernetes CRD, this must be done through the Kubernetes API server. If the application can handle all requests, the activator buffer is no longer needed. This reduces latency and increases efficiency.
The queue proxy is a sidecar container injected into the pod of the containerized application, and is always in the path of all requests. As an activator, it also sends metrics to the autoscaler and queues requests. This task becomes especially important when the Activator is not in the request path. Another thing is that the queue proxy allows the pod to shut down gracefully. Traffic already received is still handled by the user application, but incoming traffic is rejected.