Working with AWS EKS — An Introduction

Source: Google Images

There are many DevOps tools available in the industry today like Git, GitHub, Docker, Jenkins, Kubernetes, Terraform, etc. Each of these tools plays a vital role in implementing continuous integration and continuous development/deployment (CI/CD). However, to practically achieve a fast product development and deployment cycle, we need a requisite amount of resources like RAM, CPU, GPU, storage, etc., and optimized allocation and management of these resources.

Cloud computing helps us solve the demand for resources. In particular, public cloud services help us develop large scale infrastructures with reduced cost overheads and security concerns. AWS is one of the most popular public cloud service providers. AWS provides a multitude of services, ranging from operating system (OS) instances (EC2), Networking (VPC), Security (Security Groups and Firewalls), Storage (EBS, S3, EFS), Monitoring (CloudWatch), etc.

Since Kubernetes itself is a software, it is possible to run it on any computer system with an OS and other pre-requisites. Thus, it is possible to create our Kubernetes clusters (instead of using Minikube and Kubectl as a pair) on Amazon using EC2 instances. Using Kubernetes like this does not provide any additional benefits as compared to working on our local systems as we still have to handle the management of the clusters and other components. Amazon facilitates the management of Kubernetes clusters, and the resources and services of Kubernetes through the Elastic Kubernetes Service (EKS). EKS allows us to get an automatically provisioned, managed Kubernetes control plane. In both cases, we get powerful, community-backed integrations to AWS services like VPC, IAM, and service discovery as well as the security, scalability, and high-availability of AWS.

To begin working with AWS EKS we need a few pre-requisites, which are listed below:

  1. Installation of kubectl command (In my case, I have used Windows OS — Reference 3,4).
  2. Installation of AWS CLI version 2 command. (Reference 5)
  3. Installation of the official eksctl command. (Reference 6)

The installation of Minikube and kubectl go hand-in-hand even in the official documentation, and the configuration file created during the installation can be modified for our purposes, so the installation of Minikube is also preferred, although not completely necessary.

To begin working on our idea, it is best to create a separate folder as a workspace.

Creating the Cluster

The cluster of node groups can be created using a YAML file as shown in the image below. The cluster is created with the name awscluster in the Mumbai region (ap-south-1). There are two node groups, namely g1 and g2. In g1, we request the launch of 3 nodes (instances) of the t2.micro instance type. We also provide the option of SSH login with a .pem key. Similarly, in g2, we request the launch of 2 nodes of the t2.small instance type, with SSH login using the same .pem key as g1.

Figure 1: Cluster creation file

To create the cluster, we first configure an AWS profile. User profiles in AWS are created using the IAM service. Note that the AWS profile must have AdministratorAccess to create the cluster.

> aws configure
> eksctl create cluster -f aws-cluster.yml

When the cluster creation begins, the following output is seen.

Figure 2: Output for the cluster creation

We can log in to our AWS account to see how the cluster is created. In AWS, cluster creation is handled by CloudFormation. We can observe the events during the cluster creation and also check for more details if the cluster is not created due to unexpected errors.

Figure 3: CloudFormation console during cluster creation (Part 1)
Figure 4: CloudFormation console during cluster creation (Part 2)
Figure 5: Instances created for the cluster

We can check for the list of clusters we currently have using the commands:

> eksctl get clusters

Now, we need to update the config file in the .kube folder. This is the file that is used to establish a connection between the clusters and the client. The update will add the new cluster and the required details of the cluster to the config file.

> aws eks update-kubeconfig --name <cluster-name>
Figure 6: Adding the new cluster to the config file.

To see the updated config file, we use the command:

> kubectl config view

To check if we have all our nodes in Ready state, we use the command:

> kubectl get nodes

Creation of Elastic File System (EFS)

After the config file is updated, we can create our namespace and set the current (working) namespace. The creation of a namespace is not mandatory; we can work within the current default namespace as well.

> kubectl create namespace <name-of-namespace>
> kubectl config set-context --current --namespace=<name-of-namespace>

Now we can proceed with the creation of the Elastic File System (EFS). AWS provides different types of storage services like EBS, S3, and EFS, and each has its applications. EBS is a type of block storage where we can create partitions and store data. Hence, EBS is used for providing the Hard Disks of the AWS EC2 instances. However, EBS has a drawback as it cannot be mounted onto more than one instance at a given time. S3 is a type of object storage that allows mounting on multiple instances, but it is largely used for the storage of files. Editing of the data stored in S3 becomes cumbersome, especially in situations where the volume of data is large. EFS works on the principle of NFS protocol, and it allows centralized storage and sharing of files over a network.

To provide EFS storage to the nodes we have created, we need to first create an EFS using the AWS console. The EFS must be created in the VPC where the cluster is created. Also, the security groups assigned for the EFS in each availability zone must include all the security groups assigned to the nodes we have created. The security groups of the nodes can be checked in the Instances section of the AWS EC2 Dashboard.

Figure 7: Setting security groups of all nodes for the EFS in each Availability Zone
Figure 8: Successful creation of EFS

We need to note two important details about the EFS we have created — The file system ID and the DNS name. These details are important in our next step, which is the creation of the EFS provisioner.

The EFS provisioner is essentially a container that allows us to access the EFS we have created. The provisioner is created using a YAML file, whose contents are given below.

# Contents of create-efs-provisioner.yamlkind: Deployment
apiVersion: apps/v1
metadata:
name: efs-provisioner
spec:
selector:
matchLabels:
app: efs-provisioner
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: efs-provisioner
spec:
containers:
- name: efs-provisioner
image: quay.io/external_storage/efs-provisioner:v0.1.0
env:
- name: FILE_SYSTEM_ID
value: <File System ID of EFS>
- name: AWS_REGION
value: ap-south-1
- name: PROVISIONER_NAME
value: aws-cluster/aws-efs
volumeMounts:
- name: pv-volume
mountPath: /persistentvolumes
volumes:
- name: pv-volume
nfs:
server: <DNS of EFS>
path: /

Now, we can create the provisioner using the following command.

> kubectl create -f create-efs-provisioner.yaml

The last thing we need to do concerning the EFS is to install amazon-efs-utils in each of the nodes we have created by remote logging into the nodes. This can be done through the Instances section of AWS EC2 and executing the following command in each node:

$ sudo yum install amazon-efs-utils -y

Creation of Role-Based Access Control (RBAC) Authorization

The RBAC is used as a ClusterRoleBinder, where it grants permissions across the entire cluster.

The RBAC is created using a YAML file whose contents are given below.

#Contents of create-rbac.yamlapiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nfs-provisioner-role-binding
subjects:
- kind: ServiceAccount
name: default
namespace: awsns
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io

Now, we can create the RBAC using the following command.

> kubectl create -f create-rbac.yaml

Creation of StorageClass

Kubernetes offers many pre-built storage classes, but we require to provide storage from the EFS we have created. Hence, we need to create a separate storage class that can allocate storage from the EFS we have created. The StorageClass is created using the following YAML file.

#Contents of create-storage.yamlkind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: aws-efs
provisioner: aws-cluster/aws-efs

Now, we can create StorageClass using the following command.

> kubectl create -f create-storage.yaml

Testing the setup

Now, our cluster is ready for our use. We can test it out by creating a deployment that uses a custom Docker image that I have created to run HTTPD inside a container using CentOS as the operating system. We will also expose the deployment using the LoadBalancer, which is a type of Service in Kubernetes.

Firstly we need to allocate persistent storage to the deployment from the EFS, which can be done using a PersistentVolumeClaim, written in YAML format as shown below.

#Contents of httpd-storage.ymlapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: httpd-pvc
annotations:
volume.beta.kubernetes.io/storage-class: "aws-efs"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi

Next, we write the manifest file for creating our deployment and service.

#Contents of httpd-server.ymlapiVersion: apps/v1
kind: Deployment
metadata:
name: html-dp
labels:
app: html
spec:
selector:
matchLabels:
app: html
strategy:
type: Recreate
template:
metadata:
labels:
app: html
spec:
containers:
- image: akshayab99/custom-httpd:v1
name: html-con
ports:
- containerPort: 80
name: html-dp
volumeMounts:
- name: html-storage
mountPath: /var/www/html
volumes:
- name: html-storage
persistentVolumeClaim:
claimName: httpd-pvc
---apiVersion: v1
kind: Service
metadata:
name: html-svc
labels:
app: html
spec:
ports:
- port: 80
protocol: TCP
#targetPort: 80
selector:
app: html
type: LoadBalancer

Once the files are created, we can launch the PVC, deployment, and service.

> kubectl create -f httpd-storage.yml
> kubectl create -f httpd-server.yml

When all the required resources are launched, we can see the following output on the Command Line.

Figure 9: Successful launch of all required resources

To access a webpage inside the pod currently created, we need to use the external IP found as a part of the information about services. Using the external IP, we can load our webpage, as shown below.

Figure 10: Loading the test web page

Working with Helm

We have set up our cluster and launched a deployment on it. Now the management and observation of our current setup are easy as it is small in size. However, in real-world applications, the number of nodes and services being used will be much larger, which brings up the need for other tools.

Helm is a tool that can facilitate our work with Kubernetes. It acts as a package manager. Its role can be compared to the role played by pip to install python modules. Initially, Helm worked with the tool Tiller for managing Kubernetes, but the newest version has upgraded the architecture to function without Tiller. However, in this article, we will be working with the older version.

The installation of Helm, getting the list of Helm repositories, and the setting up of Tiller can be done using the following commands.

> helm init
> helm repo add stable https://kubernetescharts.storage.googleapis.com/
> helm repo list
> helm repo update
> kubectl -n kube-system create serviceaccount tiller
> kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
> helm init --service-account tiller

Helm provides many repositories to use many DevOps tools like Jenkins, Prometheus, Grafana, etc. Prometheus and Grafana are important monitoring tools that we can utilize to monitor the cluster we have created.

To set up Prometheus using Helm, we execute the following commands.

> kubectl create namespace prometheus
>helm install stable/prometheus --namespace prometheus --set alertmanager.persistentVolume.storageClass="gp2" --set server.persistentVolume.storageClass="gp2"
>kubectl get svc -n prometheus
> kubectl -n prometheus port-forward svc/<name-from-previous-cmd>-prometheus-server 9999:80

If the creation is successful, then we see the following output in the command line.

Figure 11: Launching Prometheus using Helm

We can now view the Web UI of Prometheus, as seen in the image below.

Figure 12: Prometheus Dashboard

Next, we set up Grafana, using the following commands.

> kubectl create ns grafana
> helm install stable/grafana --namespace grafana --set persistence.storageClassName="gp2" --set adminPassword=grafanapass --set service.type=LoadBalancer

Grafana requires a username and password to log in with. To find the username and password, we follow the instructions below.

Figure 13: Finding the password for the admin user of Grafana
#Command to find the password> kubectl get secret --namespace grafana romping-chipmunk-grafana -o jsonpath="{.data.admin-password}"#Decode the result above to get the password

The password is given in the Base64 format, hence we need to decode it before using it to log in into Grafana with the user as admin.

Once we have logged in to Grafan, we need to set up the dashboard needed to monitor the Kubernetes Cluster. There is a pre-built dashboard for this very purpose, hence we can import the dashboard from.

Figure 14: Importing Clustering Monitoring for Kubernetes Dashboard via https://grafana.com/ (Part 1)
Figure 15: Importing Clustering Monitoring for Kubernetes Dashboard via https://grafana.com/ (Part 2)
Figure 16: Importing Clustering Monitoring for Kubernetes Dashboard via https://grafana.com/ (Part 3)

After setting up both Prometheus and Grafana, we can monitor various parameters of the Kubernetes cluster.

Deleting the Infrastructure

Since many of the AWS components we have used are not free, it is important to remember to delete the components after use. However, deletion must also be performed to avoid any errors. Before deleting the cluster, ensure that all the Kubernetes and Helm resources are deleted, followed by the EFS storage, and then finally the cluster as a whole. Other than the EFS which must be deleted through the AWS Web UI, the other components can be deleted from the CLI itself.

> kubectl delete all --all
> kubectl delete pvc --all
> kubectl delete -f create-storage.yml
> kubectl delete all --all -n prometheus
> kubectl delete all --all -n grafana
#After deleting EFS on Web UI
> eksctl delete cluster -f aws-cluster.yml

This work has been done as a part of the 2-day training for AWS Elastic Kubernetes Service (EKS) conducted by Mr. Vimal Daga from LinuxWorld Informatics, Pvt., Ltd.

Do check out my other work with AWS!

  1. Integration of AWS, Terraform, and GitHub for Automated Deployment Infrastructure: https://medium.com/@akshayavb99/integration-of-aws-terraform-and-github-for-automated-deployment-infrastructure-da0a19ff4e86

ECE Undergrad | ML, AI and Data Science Enthusiast | Avid Reader | Keen to explore different domains in Computer Science

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store