Working with Jenkins, Docker, GitHub, and Kubernetes — Part III

Source: Google Images

Docker offers multiple advantages like rapid deployment, increased security for the deployed applications/software, optimized usage of memory and resources, etc. However, even Docker has some disadvantages, especially when it comes to the management of different containers being launched.

A major issue with using Docker alone is the absence of a fault-tolerant setup. Containers launched using Docker are not relaunched automatically. This means that if a container crashes due to any unexpected circumstances, the environment set up inside the container needs to be manually setup once again. Kubernetes allows us to manage containers efficiently. It takes care of scaling and failover for your application, provides deployment patterns, and more.

Let’s create the following setup to understand the working of basic Kubernetes resources and services better.

  1. Create a container image that’s has Jenkins installed using Dockerfiles. When we launch this image, it should automatically start the Jenkins service in the container.
  2. Create a job chain of Job1, Job2, Job3 and Job4 using build pipeline plugin in Jenkins
  3. Job 1: Pull the Github repository automatically when some developers push the repository to Github.
  4. Job 2: By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed). Expose your pod so that the testing team could perform the testing on the pod. Make the data of the pods persistent and test the code.
  5. Job 3: Test if the code is functioning correctly.
  6. Job 4: If the code does not work correctly, then send an email to the developer with error messages and redeploy the application after code is being edited by the developer.

To reach our objectives, the basic pre-requisites needed are:

  1. RHEL 8 Virtual Machine
  2. Installation of Minikube on the base system (Windows/Mac etc.)
  3. Setting up the kubectl command on the RHEL 8 Virtual machine

Launching Jenkins and Kubernetes

To work with Kubernetes, we need to first setup Minikube as well as the kubectl command in the system where we will be running Jenkins. In this setup, I will be running both kubectl command and Jenkins on a RHEL 8 VM running on the Oracle Virtual Box.

There are 2 ways with which we can work with Kubernetes. The first method is to use the command line, and the second is to use a YAML file to launch our required services in Kubernetes. The YAML file is more advantageous to use because it is possible to replicate our current setup in any system with all the required pre-requisites. In real-time applications, this allows us to replicate the testing environment for the production systems, through which the clients can access the deployed software or application.

The code we will be testing for is written in HTML, hence we will be launching a container with a HTTPD server image. However, to conform to our requirements, the image is not the default version available on Docker Hub, but a custom one where the HTTPD server runs on CentOS. To use custom images with Kubernetes, we need to first upload it to a public repository on Docker Hub. The procedure to do that is given here. The contents of the Dockerfile are given below.

Figure 1: Contents of the custom HTTPD server image

In a previous article, I have explained how to write the Dockerfile for a container image with Jenkins pre-installed. Please refer to the article here. This image is also pushed to Docker Hub for using it with Kubernetes.

Now, we write the YAML files for launching containers with Jenkins and HTTPD server. To write the YAML files, we first create a workspace, where we can store all the required files to launch our Kubernetes services. The contents of the workspace are as shown below.

Figure 2: Files to create the required containers

Amongst the files, jenkins-vol.yml and httpd-vol.yml are files that are used to allocate persistent volume to the Jenkins and HTTPD server pods. The main requirement of these persistent volumes is to ensure that the data inside the pods are not lost, even if the pods fail. More importantly, if the pods are relaunched by the deployment, the data must be still available for access and use. The contents of the files are given below.

# Contents of httpd-server.ymlapiVersion: apps/v1
kind: Deployment
metadata:
name: html-dp
labels:
env: html-dp
spec:
selector:
matchLabels:
env: html-dp
strategy:
type: Recreate
template:
metadata:
labels:
env: html-dp
spec:
containers:
- image: akshayab99/custom-httpd:v1
name: html-con
ports:
- containerPort: 80
name: html-dp
volumeMounts:
- name: html-storage
mountPath: /var/www/html
volumes:
- name: html-storage
persistentVolumeClaim:
claimName: html-pvc
---apiVersion: v1
kind: Service
metadata:
name: html-svc
labels:
env: html-svc
spec:
ports:
- port: 80
nodePort: 31203
targetPort: 80
selector:
env: html-dp
type: NodePort

The contents of httpd-vol.yml are given below.

# Contents of httpd-vol.ymlapiVersion: v1kind: PersistentVolumemetadata:
name: html-pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/root/devops_task3/jenkins/html"
---apiVersion: v1kind: PersistentVolumeClaimmetadata:
name: html-pvc
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

The contents of jenkins.yml are given below.

# Contents of jenkins.ymlapiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins-dp
spec:
replicas: 1
selector:
matchLabels:
env: jenkins
template:
metadata:
name: "jenkins-pod"
labels:
env: jenkins
spec:
containers:
- name: "jenkins-con"
image: akshayab99/jenkins-os:v3
ports:
- containerPort: 8080
name: "jenkins-dp"
volumeMounts:
- name: "jenkins-storage"
mountPath: /root
volumes:
- name: "jenkins-storage"
persistentVolumeClaim:
claimName: jenkins-pvc
---apiVersion: v1
kind: Service
metadata:
name: "jenkins-svc"
labels:
app: jenkins-svc
spec:
selector:
env: jenkins
type: NodePort
ports:
- nodePort: 31200
port: 8080
targetPort: 8080

The contents of jenkins-vol.yml are given below.

#Contents of jenkins-vol.ymlapiVersion: v1kind: PersistentVolumemetadata:
name: jenkins-pv
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/root/devops_task3/jenkins/files"
---apiVersion: v1kind: PersistentVolumeClaimmetadata:
name: jenkins-pvc
labels:
type: local
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

Before launching the respective deployments, we first create the PVC and PV for both of them using the following commands.

[root@localhost final_files]# kubectl create -f jenkins-vol.yml
[root@localhost final_files]# kubectl create -f httpd-vol.yml

Next, we launch both the deployments using the YAML files.

[root@localhost final_files]# kubectl create -f jenkins.yml
[root@localhost final_files]# kubectl create -f httpd-server.yml

Once the deployments have been launched, we can look at all the PV, PVCs, deployments, services, and pods currently active using the following command

[root@localhost final_files]# kubectl get all

The output of the command is shown below.

Figure 3: Currently active services, PVs, and PVCs in Kubernetes

In the outputs, we can see that there are pods, deployments, and ReplicaSets launched for both Jenkins and HTTPD images, even though we did not explicitly request for the launch of ReplicaSets. This is because, deployments work using ReplicaSets in the background, which is why they can launch multiple pods (replicas) of the same kind with the same label as mentioned in the template section of the YAML file. This brings up a question — why use deployments instead of ReplicaSets?

Deployments are more powerful because they allow us to use different rolling updates strategies. This essentially means that they allow us to launch different versions of the same application. Thus, we can ‘rollback’ to previous versions if the current one is not up to expectations.

Now, we can set up the admin user in Jenkins and start to create the jobs.

Jenkins Jobs

Job1 of Jenkins requires us to pull code from the GitHub repository automatically. This requires us to set up a Webhook with the Jenkins URL, which I have described as a part of a previous article. The configurations for Job1 are given below.

Figure 4: Job 1 in Jenkins (Part 1)
Figure 5: Job 1 in Jenkins (Part 2)
Figure 6: Job 1 in Jenkins (Part 3)

Job 2 in Jenkins is meant to shift code to the required container (inside a pod) based on the language of the code. The configurations of Job 2 are given below.

Figure 7: Job 2 in Jenkins (Part 1)
Figure 8: Job 2 in Jenkins (Part 2)

Job 3 in Jenkins is meant to check if the code is functioning correctly. The configurations for Job 3 are given below.

Figure 9: Job 3 in Jenkins (Part 1)
Figure 10: Job 3 in Jenkins (Part 2)

Since we perform job linking to automate the whole task, we can use a build pipeline as shown below. The green color indicates that all the jobs have been built successfully.

Figure 11: Build Pipeline view of the Jenkins Jobs

The website output is very simple. It is purely meant to test functionalities of all the jobs we have built.

Figure 12: Website used for testing

The specialty of our setup is the fact that, even if the pods get deleted or crash down due to unexpected circumstances, the webpage will still be accessible with the same content, as we have provided persistent storage to the HTTPD deployment and the use of deployments also ensures that if any pod crashes or is deleted, another pod is immediately launched based on the number of replicas we have requested for in the YAML file.

This article is written as a part of the DevOps Assembly Lines Training Program conducted by Mr. Vimal Daga from LinuxWorld Informatics Pvt., Ltd.

You can find some of my previous works for the DevOps Assembly Lines Program below.

  1. Working with Jenkins — An Introduction: https://medium.com/@akshayavb99/working-with-jenkins-an-introduction-48ecf3de3c25
  2. Working with Jenkins, Docker, Git, and GitHub — Part II: https://medium.com/@akshayavb99/working-with-jenkins-docker-git-and-github-part-ii-d74b6e47140c

ECE Undergrad | ML, AI and Data Science Enthusiast | Avid Reader | Keen to explore different domains in Computer Science

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store