Working with Jenkins, Docker, Git, and GitHub — Part II
I had previously written an article where I gave a glimpse into how Jenkins can be used to automate the product development, deployment, monitoring, and maintenance cycle. This article is going to be along the same lines, but with a different system being set up.
In this article, we can see the integration of Jenkins, Docker, Git, and GitHub with the help of the following objectives:
- Create a Docker container image that has Jenkins installed using Dockerfile. When we launch this image, it should automatically start the Jenkins service in the container.
- Create a job chain of job1, job2, job3, and job4 using the build pipeline plugin in Jenkins.
- Job1: Set up a local Git repository. When the developers push the local repository to GitHub, Jenkins Job 1 must automatically pull the GitHub repository to the localhost.
- Job 2: Job 2 must launch specific containers depending on the type of code in each file. For example, if the code is in PHP, then Jenkins must start the container that has PHP installed in it during the respective Dockerfile creation and build.
- Test if the code file is working or not. If the code is not working, then an email notification is sent to the developer with error messages.
- Job 3: If the container running the code collapses then Job 3 must automatically start the container again with the code file.
To accomplish these objectives, the basic pre-requisites we need are:
- A RHEL8 OS launched as a VM. (I use Oracle Virtual Box)
- Installation of Git (In the RHEL8 VM or bare metal OS) and Docker (in RHEL8 VM)
Creation of Docker Container Images using Dockerfile
The first objective is to create all the container images that can come into use for us. Currently, the type of code files I am considering are HTML (.html), and Python (.py) files. Hence, we need to create four Dockerfile for launching Jenkins, HTML, and Python files each.
There are multiple methods to create custom Docker container images, but the method we will use is the creation of Dockerfiles. To create a Dockerfile, we first need to set up a workspace from the command line.
# mkdir test-systemd
# cd test-systemd
# gedit Dockerfile
The commands used are for the following purpose:
FROM centos- It references the base OS for the new image.
RUN yum install wget -y- Installation of wget command (Used to install Jenkins)
RUN yum install net-tools -y- Installation of net-tools which form the base of Linux OS networking
RUN wget -0 /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
RUN rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key- Commands 4. and 5. are meant for the installation of Jenkins. (Refer to the link https://www.jenkins.io/download/ )
RUN yum install java -y && yum install jenkins -y- The installation of Jenkins and Java as Jenkins is dependant on Java for its working.
RUN yum install git -y- Installation of Git to perform commands by interacting with GitHub repositories
RUN yum install gedit -y- Installation of text editor Gedit for performing minor changes in the configuration files of Jenkins to enable email services. (Can be ignored)
RUN yum install openssh-clients -y— The installation allows us to use remote access of other systems from within the container and vice-versa.
RUN echo -e "jenkins ALL=(ALL) NOPASSWD:ALL" /etc/sudoers- This allows Jenkins to have maximum access to perform commands by modifying the sudoers file.
After saving the Dockerfile, we use
docker build to save the container image.
# docker build -t jenkins-os:v3 .
# docker run -dit -p 8888:8080 -v /task2:/task2 --name jen-os jenkins-os:v3
# docker exec jen-os sudo cat /var/lib/jenkins/.jenkins/secrets/initialAdminPassword
After we create the docker container image under the name,
jenkins-os:v3, we start a container with this image. The options
-dit means that there is an interactive terminal running in the background once the container is started. To allow exposure to the outside world, the port 8888 is exposed using PATing and a volume is mounted from the host system to ensure data persistence.
When we launch Jenkins in the web browser using the URL <container IP>:<Port>, we see the installation page of Jenkins, where we can use the
initialAdminPassword to login and setup the admin user account. Once we have logged in as the admin user, we need to install several Plugins in the Manage Plugins section of Manage Jenkins option. The plugins which must be installed are:
- Github Plugin
- Build Pipeline Plugin
- Delivery Pipeline Plugin
- Email Extension Plugin
- SSH Plugin
- Publish Over SSH Plugin
- SSH Agent Plugin
- SSH2 Easy Plugin
Once the plugins are installed, reload Jenkins on the Web UI and to update the list of installed plugins.
We need to enable the communication between the host RHEL8 VM and the Jenkins container using SSH. To achieve this, we need to set up the credentials of RHEL8 as a remote SSH host in the Configure System section of Manage Jenkins.
Since our set up also involves pulling data from Github, we can configure the Webhooks for the GitHub repository as well. This configuration is done on both Jenkins as well as GitHub.
- In Jenkins, we set up a Webhook as per the screenshot given below, in the Configure Systems section of Manage Jenkins.
2. In GitHub, go to Settings and set up the Webhook as shown below.
Once all the settings are finished, we can proceed to the jobs in Jenkins.
The Dockerfile and the container image for the containers for Python files can be created similarly. The contents of the Dockerfile is as shown below.
Job 1 asks us to pull the content from the online GitHub repository. The process of using both Git and GitHub repositories and pulling the content from the GitHub repository has been discussed in a previous article, and we will be following the same procedure here.
In Figure 8, we have the main shellcode that Job 1 will execute during a successful build. The code essentially copies only the files with the extensions .html and .py into the given directory task2.
In Job 2, we execute shell commands remotely on the host system using SSH. There are two build shell scripts, one for each type of file being handled. The first build shell is involved in launching the container for .html files. The second shell script is involved in loading the .py files and launching the required container.
In Jenkins Job 3, we are testing the file(s) to see if they are executed successfully. Hence, we write a remote shell script under Builds for the .html file(s) (Figure 14) and the .py file(s) (Figure 15). Also, the Plugins we have installed for Email Notifications allow us to send emails if the build of Job 2 is unsuccessful.
In Job 4, the containers are monitored to see if they stop running due to any unexpected reason. If they have stopped, then they are restarted again. This monitoring has been configured using a periodic build, where the Job is built every 15 minutes as shown in Figure 16.
Based on the set up we have completed, we can see the following outputs