The CloudStack Kubernetes Services (CKS) uses CoreOS templates to deploy Kubernetes clusters. However, as CoreOS reached EOL on May 26th, 2020 we needed to find a suitable replacement meeting the requirements of resilience, security, and popularity in the community. Keeping these requirements in mind, we have chosen to modify the existing Debian-based SystemVM template so it can also be used by CKS instead of CoreOS.
Before coming to this decision, we considered other operating systems, such as FlatCar Linux, Alpine Linux and Debian, and based our decision on the following parameters:
| FlatCar Linux | Alpine Linux | Debian | |
| Brief Description | Drop-in replacement for CoreOS | Alpine Linux is a Linux distribution based on musl and BusyBox, designed for security, simplicity, and resource efficiency | Debian is one of the oldest operating systems based on the Linux kernel. New distributions are updated regularly, and the next candidate is released after a time-based freeze. |
| Size | ~ 500MB – 600MB | Small image of approx. 5MB – Because of its small size, it is commonly used in containers providing quick boot-up times | ~ 500MB – 600MB |
| Security | Quite secure as it mitigates security vulnerabilities by means of delivering the OS as an immutable filesystem | All userland binaries are compiled as Position Independent Executables (PIE) with stack smashing protection. These proactive security features prevent exploitation of entire classes of zero-day and other vulnerabilities. | Debian is on a par with most other Linux distributions. |
| Release Management | Frequent releases – almost bi-weekly or monthly | There are several releases of Alpine Linux available at the same time. There is no fixed release cycle but typically every 6 months | Debian announces its new stable release on a regular basis. 3 years of full support for each release and 2 years of extra LTS support. |
| Maintenance | It is maintained by Kinvolk – a Berlin based consulting firm known for their work around rkt, kubernetes, etc. | Alpine linux is backed by a pretty large community base with mailer lists, etc to find support | Unparalleled support –claim to provide you with answers for queries on mailing lists within minutes! |
| Main Reason for Choosing / Not Choosing | NOT CHOSEN: A small community, not a popular choice and chances of meeting the same fate as CoreOS i.e., EOL | NOT CHOSEN: Init system used by Alpine Linux is openrc – and up until recently k8s did not support openrc systems https://github.com/kubernetes/kubeadm/issues/1295 |
CHOSEN: Huge community support, and most importantly – we can modify the existing systemVM templates! |
Using the modified System VM template also simplifies the use of CKS. Using CoreOS to deploy Kubernetes clusters in CKS, we needed to first register the CoreOS template and ensure that the template name coincided with the name set against the global settings shown below. However, with the new Debian-based SystemVM templates, this is no longer necessary, and these global settings are not required:
To ensure the new SystemVM template will support deployment of Kubernetes clusters, we have included docker, containerd and cloud-init packages, which will only be enabled on SystemVM types for CKS nodes (as these packages are only used by the CKS nodes). These services are disabled on all other SystemVM hypes.
So that we do not increase the overall size of the SystemVM templates, we have included support for growing / resizing the root disk partition during boot up to a predefined / provided disk size. For CKS nodes, the minimum root disk size will be 8GB and can be increased by setting the node root disk size while creating the Kubernetes cluster. For other systemVMs the root disk size can be configured by setting the ‘systemvm.root.disk.size’ global setting
In summary, from Apache CloudStack 4.16 LTS onwards, CKS will use the modified (Debian) SystemVM templates for deployment of Kubernetes clusters.
Pearl is a quality and technology driven software engineer, with 5 years’ experience with relevant expertise in providing solutions to the telecom and software industry. Pearl has an excellent grasp of the evolving technologies in the changing telecom space. She is a go-getter, with a flair for learning new technologies. Pearl is based in Bangalore, India. Self-learning and self-motivation are the mantras that she follows to keep herself abreast with new things in her field of work.
