Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.
How to setup Kubernetes
The central component of Kubernetes is a cluster, which is itself made up of multiple physical or virtual machines. Each cluster component performs a specific function as either a master or a workerâa master controls and manages the containers in the nodes, while a worker hosts the groups of one or more containers.
The diagram below portrays a general framework for a Kubernetes cluster:
Kubernetes Architecture
This feature-rich cluster contains six key components, namely an API server, scheduler, controller, etcd, kubelet, and kube-proxy. Through this lens, the first four components run on the master, whereas the rest of the functions run on the worker.
In this article, youâll learn how to develop a Kubernetes cluster from scratch.
Minikube and Kubeadm
When it comes to designing a cluster, minikube and kubeadm are the predominant solutions. Designed for learning, development and testing, minikube is a fast and simple solution for deploying a single node cluster.
Architecture overview of minikube
kubeadm builds a minimum viable, production-ready Kubernetes cluster that conforms to best practices. It also allows us to choose the container runtime, though it has Docker by default. This solution requires a minimum of two VMs to run the master and worker.
Overview of kubeadm architecture
Now, letâs delve into the installation steps. Theyâll be illustrated from the root user-space, so if youâre trying them from non-root user-space, add sudo before every command.
Container Runtime: Docker Installation
Using Docker as the default container runtime, letâs start the installation on the machine.
First, update the existing repository in the machine with the following command:
apt-get update -y
Next, install a few prerequisite packages. These allow apt to use secure channels using HTTPS:
apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y
Then, add the GPG key from the official Docker repository:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg âdearmor -o /usr/share/keyrings/docker- archive-keyring.gpg
Add the Docker repository to the apt sources list:
echo âdeb [arch=$(dpkg âprint-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stableâ | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Next, update the package information with Docker packages from the apt sources list:
apt-get update -y
Finally, install Docker:
apt-get install docker-ce docker-ce-cli containerd.io -y
Update the default Docker daemon configuration in /etc/docker/daemon.json:
cat <<EOF | sudo tee /etc/docker/daemon.json
{
âexec-optsâ: [ânative.cgroupdriver=systemdâ],
âlog-driverâ: âjson-fileâ,
âlog-optsâ: {
âmax-sizeâ: â100mâ
},
âstorage-driverâ: âoverlay2â
}
Enable the process to start on boot, and reload the Docker daemon:
systemctl enable docker
systemctl daemon-reload
systemctl restart docker
To get the present status of the Docker daemon installed in your machine, use:
systemctl status docker
â docker.service â Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Fri <date> <time> UTC; <time> ago
..
.. output truncated ...
..
Now, letâs render the version information in an easy-to-read layout:
docker âversion
Output: Docker version <version>, build <build>
If we want to access the Docker from non-root user-space, create a new Docker group and add the current userspace to it:
usermod -aG docker $USER
# groupadd docker
Weâll then run a hello-world image to check every component is working:
docker run hello-world
Output: Hello from Docker!
This message shows that your installation appears to be working correctly.
Congrats! Docker is successfully running on your machine.
Standalone Kubernetes Cluster: minikube
Minikube is the local Kubernetes cluster used primarily for learning and exploration of Kubernetes. We need a simple Docker or VM environment in a Linux, macOS, or Windows platform. In our case, Docker is the container runtime for our Kubernetes cluster, and you can use the Docker installation instructions from the previous section.
Assuming youâve successfully installed Docker, letâs start the minikube installation.
First, download the latest minikube library:
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
Use this install command to copy the file and set the file attributes for execution:
install minikube-linux-amd64 /usr/local/bin/minikube
Finally, with Docker as the container runtime, letâs start the minikube:
minikube start âdriver=docker
đ minikube v1.24.0 on Ubuntu 20.04 (amd64)
⨠Using the docker driver based on existing profile
đ Starting control plane node minikube in cluster minikube
đ Pulling base image âŚ
𤡠docker âminikubeâ container is missing, will recreate.
đĽ Creating a docker container (CPUs=2, Memory=2200MB) âŚ
đł Preparing Kubernetes v1.22.3 on Docker 20.10.8 âŚ
đ Done! kubectl is now configured to use âminikubeâ cluster and âdefaultâ namespace by default
Letâs render the minikube version in an easy-to-read layout:
minikube version
minikube version: v1.24.0
commit: 76b94fb3c4e8ac5062daf70d60cf03ddcc0a741b
Create an alias for the âeasy access to kubectlâ command:
alias kubectl=âminikube kubectl ââ
To get the list of all running pods in the cluster, use:
# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-minikube 1/1 Running 1 (6m58s ago) 7h35m
kube-system kube-apiserver-minikube 1/1 Running 1 (6m58s ago) 7h35m
kube-system kube-controller-manager-minikube 1/1 Running 1 (6m58s ago) 7h35m
kube-system kube-proxy-qjnsk 1/1 Running 1 (6m58s ago) 7h35m
kube-system kube-scheduler-minikube 1/1 Running 1 (6m58s ago) 7h35m
To get the cluster node details, use:
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready control-plane, master 7h52m v1.22.3
Congrats! You just successfully built a minikube cluster On the local machine.
Production-Grade Kubernetes Cluster: kubeadm
The kubeadm toolbox acts as a bootstrap for creating a production-grade Kubernetes cluster, making a single master node with an etcd configuration. Here, Docker is the clusterâs container runtime, and you can refer to our Docker installation steps above.
Assuming youâve successfully installed Docker, letâs build the cluster using the kubeadm toolbox.
Add the Kubernetes repo GPG key into the key manager:
# curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
Add the Kubernetes repo details to the repository source list:
# echo âdeb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial mainâ | sudo tee /etc/apt/sources.list.d/kubernetes.list
Here, the installation comes in three pieces: kubelet, kubeadm, and kubectl. Letâs update the repository and start the installation:
# apt update -y
# apt-get install -y kubelet kubeadm kubectl
To hold the installed packages with current versions, use the command below. These packages cannot be upgraded or removed until the hold mark is removed.
# apt-mark hold kubelet kubeadm kubectl
Enable the process to start on boot and reload the kubelet daemon. Check the current status of the daemon:
# systemctl enable kubelet
# systemctl daemon-reload
# systemctl restart kubelet.service
Kubeadm installation is now complete. Letâs create the new cluster using kubeadm init:
# kubeadm init
[init] Using Kubernetes version: v1.23.1
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
# export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run âkubectl apply -f [podnetwork].yamlâ
We often use Calico as the clusterâs network provider, as shown here:
# kubectl create -f https://docs.projectcalico.org/v3.15/manifests/calico.yaml
configmap/calico-config
createdclusterrole.rbac.authorization.k8s.io/calico-kube-controllers createdclusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers
createdserviceaccount/calico-kube-controllers created
Now, we can join many worker nodes by running the following join command on each worker node as root:
# kubeadm join 10.190.0.2:6443 âtoken ls7ryl.mlrc2oaoob15gbog âdiscovery-token-ca-cert-hash sha256:13aa6e021229373bfdceb8537ab50056c7fb6b4b67435ad165ceb1b5131c9dfc
[preflight] Running pre-flight checks
[preflight] Reading configuration from the clusterâŚ
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS BootstrapâŚ
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Letâs execute the below command to check the node joining status on the cluster:
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
iod-master Ready control-plane,master 44h v1.23.1
iod-worker-2 NotReady <none> 32h v1.23.1
iod-worker-3 Ready <none> 32s v1.23.1
Kubernetes Cluster Alternatives
kOps (Kubernetes Operations) is an open-source tool that helps us create and manage the Kubernetes cluster. Itâs also the best tool for deploying a small-scale, production-grade cluster, allowing the deployment of highly available Kubernetes clusters on cloud platforms like AWS and GCP.
Some of kOps striking, out-of-the-box features include: CloudFormation and Terraform configuration, automatic idempotency, rolling cluster updates, and master high availability.
Kind is the tool for deploying a less resource-intensive cluster locally. Again, itâs a single-node cluster running on a Docker container. We can quickly deploy the Kubernetes cluster using a simple YAML file. However, the network management for this option is a little complex for beginners.
Another simplified Kubernetes cluster solution for IoT and edge computing is K3s. Designed for production workloads inside the IoT appliances, the total package weighs less than 50MB. This further reduces dependency and the number of steps involved in the lifecycle management of the cluster. It works on a range of devices, from a Raspberry PI to a Cloud platform, and itâs optimized for the ARM platform.
Finally, the click-and-run solution for deploying a Kubernetes cluster locally and running in under 60 seconds is MicroK8s by Canonical. It provides an inexpensive ground for proving ideas and with lower overhead, developers can ship builds faster than ever.
Secure the Cluster with Kubescape by ARMO
In the digital era, establishing security guardrails is essential for safeguarding your devices from attackers. Cluster hardening is one of the most critical processes for securing the configuration and settings of the cluster, to reduce IT vulnerability and the probability of being compromised. It also reduces the attack surface and vectors for attackers who continuously seek to exploit devices for malicious activity.
Kubescape scans K8s clusters, YAML files, and HELM charts, detecting misconfigurations according to multiple frameworks (such as the NSA-CISA, MITRE ATT&CKÂŽ), software vulnerabilities, and RBAC (role-based-access-control) violations at early stages of the CI/CD pipeline, calculates risk score instantly and shows risk trends over time.
You can quickly install Kubescape using the command:
# curl -s https://raw.githubusercontent.com/armosec/kubescape/master/install.sh | /bin/bash
After successful installation, check the Kubescape version:
# kubescape version
Output: Your current version is: v1.0.90
Now letâs scan the cluster as per the NSA framework:
# kubescape scan framework nsa
We can also scan the cluster as per the MITRE framework just by changing the name:
# kubescape scan framework mitre
We can skip the namespaces using the command:
# kubescape scan framework nsa âexclude-namespaces kube-system,kube-public
Kubescape's scan results summary
# kubescape scan framework nsa âinclude-namespaces development,staging,production
# kubescape scan framework nsa *.yaml
# kubescape scan framework nsa âverbose
# kubescape scan framework nsa âformat json âoutput results.json
Besides the cluster setup, we can scan the container images at frequent intervals, theyâll then be uploaded into the Kubescape SaaS platform. The scan runs every midnight according to the default schedule, but we can change the schedule by updating the scanSchedule Helm value in the armo-scan-scheduler.
The Kubescape SaaS platform comes with enhanced security and is easy to use. We can easily tunnel the cluster scan results to the cloud for a detailed review.
Summary
Today, weâve covered the basics of Kubernetes clusters and their core components. Weâve also seen an architectural overview of kubeadm and minikube, a step-by-step guide for building the cluster, as well as a number of tools available for cluster development.
With so many ways to build the cluster, Kubescape provides a comprehensive security posture and quick installation.
Written by River
CEO @ AIO Tech Ltd
CEO @ Seven Seas Hosting