08 Jan 2022
In this article I discuss some of the fundamentals of Kubernetes its core components and architecture including Master and Worker nodes. I then introduce MicroK8s and step through how we can install and configure a 3-node cluster with ease.
With the rise of containerisation, managing 100s or 1000s of containers was difficult. A proper set of tools were required to help with orchestration, enter Kubernetes.
Kubernetes can be used on physical servers, virtual servers and cloud infrastructure.
A cluster is made up of at least 1 master node (2+ masters for production), worker nodes and a virtual network.
Worker nodes have a kubelet process running on them, which enables cluster communication and the ability for the master to execute tasks on nodes.
Worker nodes container docker containers, which our applications and workloads run on.
Master nodes contain critical processes to enable running and managing our cluster such as:
Lastly the Virtual Network, which creates a unified machine with the sum of all resources.
A pod is an atomic unit, a wrapper around our container(s), they are self contained with their own internal IP address.
Administrators do not create containers, they only work with pods. A pod usually runs relatively tightly coupled applications.
When pods are “restarted”, a new pod is created to replace the old one. This new pod also gets a new internal IP address.
A Service is an abstraction which defines a logical set of Pods and a policy by which to access them.
Services allow your applications to receive traffic.
Services can be exposed in different ways by specifying a
type in the ServiceSpec:
An ClusterIP Service (Internal) sits in front of the pod, therefore when a pod is renewed the service is still in tack and nothing breaks. A service also acts as:
A NodePort Service opens a service externally to the Kubernetes cluster, this would provide a node IP address and port number.
A Load Balancer Service
An ExternalName Service
Ingress (Reverse Proxy) is used to route traffic into the Kubernetes cluster, which sits in front of the service providing a secure connection and domain names for offered services.
Configuration map usually contains URLs of a database, which can be connected to a pod. When a URL changes you simply update the map instead of rebuilding the whole pod.
Secret is a place for storing credentials, certificates or other sensitive data, which are stored unencrypted in a BASE64 encoded format, which can be encrypted by third party tools.
Note: Secrets need to be manually encoded into BASE64, this is not an automated process.
echo -n mysecretusername | base64 echo -n mysecretpassword | base64
As in containers volumes are used to store data to make it persistent through renewals.
Deployments or Blueprints are used to create the services, pods and replicas.
Deployments are an abstraction over pods.
Databases cannot be replicated using Deployments as they maintain a state. To avoid data corruption of data we can use StatefulSet, to handle scaling of the database and synchronising of data between databases.
StatefulSets are complex and it is often seen to host databases outside of Kubernetes.
Configurations are made up of 3 parts
Configuration files are normally stored along side your code in a version controlled setting such as Git.
EXAMPLE KUBERNETES CONFIGURATION FILES
ConfigMap - mongo-config.yaml
apiVersion: v1 kind: ConfigMap metadata: name: mongo-config data: mongo-url: mongo-service
Secret - mongo-secret.yaml
apiVersion: v1 kind: Secret metadata: name: mongo-secret type: Opaque data: mongo-user: dXNlcm5hbWU= mongo-password: QDg1MDN3MHJk
apiVersion: apps/v1 kind: Deployment metadata: name: mongo-deployment # Labels are Key value pairs, used to identify components labels: app: mongo spec: # How many Pods to create replicas: 1 # MatchLabels are used to match pods to deployments in Kubernetes selector: matchLabels: app: mongo # Blueprint for Pods - Configures the Pod template: metadata: labels: app: mongo spec: containers: - name: mongodb image: mongo:5.0 ports: - containerPort: 27017 # Environment Variables from Secrets env: - name: MONGO_INITDB_ROOT_USERNAME valueFrom: secretKeyRef: name: mongo-secret key: mongo-user - name: MONGO_INITDB_ROOT_PASSWORD valueFrom: secretKeyRef: name: mongo-secret key: mongo-password --- apiVersion: v1 kind: Service metadata: name: mongo-service spec: # Used to find the Pods related to Deployment selector: app: mongo ports: - protocol: TCP # Endpoint Port port: 8080 # Port of the Pods targetPort: 27017
apiVersion: apps/v1 kind: Deployment metadata: name: webapp-deployment # Labels are Key value pairs, used to identify components labels: app: webapp spec: # How many Pods to create replicas: 1 # MatchLabels are used to match pods to deployments in Kubernetes selector: matchLabels: app: webapp # Blueprint for Pods - Configures the Pod template: metadata: labels: app: webapp spec: containers: - name: webapp image: nanajanashia/k8s-demo-app:v1.0 ports: - containerPort: 3000 # Environment Variables from Secrets and ConfigMap env: - name: USER_NAME valueFrom: secretKeyRef: name: mongo-secret key: mongo-user - name: USER_PWD valueFrom: secretKeyRef: name: mongo-secret key: mongo-password - name: DB_URL valueFrom: configMapKeyRef: name: mongo-config key: mongo-url --- apiVersion: v1 kind: Service metadata: name: webapp-service spec: # External Service type: NodePort # Used to find the Pods related to Deployment selector: app: webapp ports: - protocol: TCP # Endpoint Port port: 3000 # Port of the Pods targetPort: 3000 # nodePort has a pre-defined ranged 30000-32767 nodePort: 30100
Creating the files in Kubernetes
kubectl apply -f mongo-config.yaml kubectl apply -f mongo-secret.yaml kubectl apply -f mongo.yaml kubectl apply -f webapp.yaml
kubectl get all kubectl get configmap kubectl get secret
Checking the logs of the Pod
kubectl logs <pod name>
Accessing the Service
kubectl get service kubectl get node -o wide
Ubuntu is the world’s most popular cloud operating system across public cloud.
Light weight, Minimal footprint
Over the Air Updates and Security fixes
MicroK8s can run anywhere from:
Install the snap package
sudo snap install microk8s --classic
Add your user to the MicroK8s group
sudo usermod -a -G microk8s $USER sudo chown -f -R $USER ~/.kube newgrp microk8s
Inspect the installed and running services - useful for troubleshooting
built-in tools - use tab for auto suggestions
such as cluster information
A full list of services and ports can be found on the MicroK8s website.
Firewall rules for ports:
If we were to use UFW we would add allow rules.
sudo ufw allow 16443/tcp sudo ufw allow 10250/tcp sudo ufw allow 10255/tcp sudo ufw allow 25000/tcp sudo ufw allow 12379/tcp sudo ufw allow 10257/tcp sudo ufw allow 10259/tcp sudo ufw allow 19001/tcp
Obtain add node information
On one node, run the given command
From the node you wish to join to this cluster, run the following: microk8s join 192.168.1.82:25000/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
After the first node completes, verify using
microk8s.kubectl get nodes
Then go through the process again, starting from
We’ll need a new add-node token per host we add, its also advised to only add one node at a time.
Once we have all our three nodes clustered we can view the MicroK8s status to see if the automatic High-Availability has been enabled
From the figure we can see that the ha-cluster is enabled.
Verify the cluster
Enable built-in workloads - addons
microk8s.enable dashboard dns ingress rbac
dashboard: The standard Kubernetes Dashboard.
dns: Deploys CoreDNS. It is recommended you always enable it.
ingress: A simple ingress controller for external access, similar to how reverse proxies work.
rbac: Enable Role Based Access Control for authorisation.
registry: Deploy a private image registry and expose it on localhost:32000.
Add-ons will be installed across all nodes within the cluster, so there is no requirement to run the enable command on each node.
As shown below, we can see each node being configured from the single command.
MetalLB will prompt for a pool of addresses
Enabling MetalLB Enter each IP address range delimited by comma (e.g. '10.64.140.43-10.64.140.49,192.168.0.105-192.168.0.111'):
microk8s.kubectl get pods --all-namespaces
pods will take time to finish installing, refresh the
get pods commands to see the progress.
microk8s.kubectl get services --all-namespaces
Create a proxy to your nodeIP
microk8s.kubectl port-forward -n kube-system service/kubernetes-dashboard 10443:443 --address 192.168.1.60
Visit your Node IP address and port number
Enter your Token
And you will be presented with the Kubernetes Dashboard.