Jul 11, 2020
Working with AWS EKS — An Introduction
There are many DevOps tools available in the industry today like Git, GitHub, Docker, Jenkins, Kubernetes, Terraform, etc. Each of these tools plays a vital role in implementing continuous integration and continuous development/deployment (CI/CD). However, to practically achieve a fast product development and deployment cycle, we need a requisite amount of resources like RAM, CPU, GPU, storage, etc., and optimized allocation and management of these resources.
Cloud computing helps us solve the demand for resources. In particular, public cloud services help us develop large scale infrastructures with reduced cost overheads and security concerns. AWS is one of the most popular public cloud service providers. AWS provides a multitude of services, ranging from operating system (OS) instances (EC2), Networking (VPC), Security (Security Groups and Firewalls), Storage (EBS, S3, EFS), Monitoring (CloudWatch), etc.
Since Kubernetes itself is a software, it is possible to run it on any computer system with an OS and other pre-requisites. Thus, it is possible to create our Kubernetes clusters (instead of using Minikube and Kubectl as a pair) on Amazon using EC2 instances. Using Kubernetes like this does not provide any additional benefits as compared to working on our local systems as we still have to handle the management of the clusters and other components. Amazon facilitates the management of Kubernetes clusters, and the resources and services of Kubernetes through the Elastic Kubernetes Service (EKS). EKS allows us to get an automatically provisioned, managed Kubernetes control plane. In both cases, we get powerful, community-backed integrations to AWS services like VPC, IAM, and service discovery as well as the security, scalability, and high-availability of AWS.
To begin working with AWS EKS we need a few pre-requisites, which are listed below:
- Installation of kubectl command (In my case, I have used Windows OS — Reference 3,4).
- Installation of AWS CLI version 2 command. (Reference 5)
- Installation of the official eksctl command. (Reference 6)
The installation of Minikube and kubectl go hand-in-hand even in the official documentation, and the configuration file created during the installation can be modified for our purposes, so the installation of Minikube is also preferred, although not completely necessary.
To begin working on our idea, it is best to create a separate folder as a workspace.
Creating the Cluster
The cluster of node groups can be created using a YAML file as shown in the image below. The cluster is created with the name awscluster in the Mumbai region (ap-south-1). There are two node groups, namely g1 and g2. In g1, we request the launch of 3 nodes (instances) of the t2.micro instance type. We also provide the option of SSH login with a .pem key. Similarly, in g2, we request the launch of 2 nodes of the t2.small instance type, with SSH login using the same .pem key as g1.
To create the cluster, we first configure an AWS profile. User profiles in AWS are created using the IAM service. Note that the AWS profile must have AdministratorAccess to create the cluster.
> aws configure
> eksctl create cluster -f aws-cluster.ymlWhen the cluster creation begins, the following output is seen.
We can log in to our AWS account to see how the cluster is created. In AWS, cluster creation is handled by CloudFormation. We can observe the events during the cluster creation and also check for more details if the cluster is not created due to unexpected errors.
We can check for the list of clusters we currently have using the commands:
> eksctl get clustersNow, we need to update the config file in the .kube folder. This is the file that is used to establish a connection between the clusters and the client. The update will add the new cluster and the required details of the cluster to the config file.
> aws eks update-kubeconfig --name <cluster-name>
To see the updated config file, we use the command:
> kubectl config viewTo check if we have all our nodes in Ready state, we use the command:
> kubectl get nodesCreation of Elastic File System (EFS)
After the config file is updated, we can create our namespace and set the current (working) namespace. The creation of a namespace is not mandatory; we can work within the current default namespace as well.
> kubectl create namespace <name-of-namespace>
> kubectl config set-context --current --namespace=<name-of-namespace>Now we can proceed with the creation of the Elastic File System (EFS). AWS provides different types of storage services like EBS, S3, and EFS, and each has its applications. EBS is a type of block storage where we can create partitions and store data. Hence, EBS is used for providing the Hard Disks of the AWS EC2 instances. However, EBS has a drawback as it cannot be mounted onto more than one instance at a given time. S3 is a type of object storage that allows mounting on multiple instances, but it is largely used for the storage of files. Editing of the data stored in S3 becomes cumbersome, especially in situations where the volume of data is large. EFS works on the principle of NFS protocol, and it allows centralized storage and sharing of files over a network.
To provide EFS storage to the nodes we have created, we need to first create an EFS using the AWS console. The EFS must be created in the VPC where the cluster is created. Also, the security groups assigned for the EFS in each availability zone must include all the security groups assigned to the nodes we have created. The security groups of the nodes can be checked in the Instances section of the AWS EC2 Dashboard.
We need to note two important details about the EFS we have created — The file system ID and the DNS name. These details are important in our next step, which is the creation of the EFS provisioner.
The EFS provisioner is essentially a container that allows us to access the EFS we have created. The provisioner is created using a YAML file, whose contents are given below.
# Contents of create-efs-provisioner.yamlkind: Deployment
apiVersion: apps/v1
metadata:
name: efs-provisioner
spec:
selector:
matchLabels:
app: efs-provisioner
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: efs-provisioner
spec:
containers:
- name: efs-provisioner
image: quay.io/external_storage/efs-provisioner:v0.1.0
env:
- name: FILE_SYSTEM_ID
value: <File System ID of EFS>
- name: AWS_REGION
value: ap-south-1
- name: PROVISIONER_NAME
value: aws-cluster/aws-efs
volumeMounts:
- name: pv-volume
mountPath: /persistentvolumes
volumes:
- name: pv-volume
nfs:
server: <DNS of EFS>
path: /
Now, we can create the provisioner using the following command.
> kubectl create -f create-efs-provisioner.yamlThe last thing we need to do concerning the EFS is to install amazon-efs-utils in each of the nodes we have created by remote logging into the nodes. This can be done through the Instances section of AWS EC2 and executing the following command in each node:
$ sudo yum install amazon-efs-utils -yCreation of Role-Based Access Control (RBAC) Authorization
The RBAC is used as a ClusterRoleBinder, where it grants permissions across the entire cluster.
The RBAC is created using a YAML file whose contents are given below.
#Contents of create-rbac.yamlapiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nfs-provisioner-role-binding
subjects:
- kind: ServiceAccount
name: default
namespace: awsns
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
Now, we can create the RBAC using the following command.
> kubectl create -f create-rbac.yamlCreation of StorageClass
Kubernetes offers many pre-built storage classes, but we require to provide storage from the EFS we have created. Hence, we need to create a separate storage class that can allocate storage from the EFS we have created. The StorageClass is created using the following YAML file.
#Contents of create-storage.yamlkind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: aws-efs
provisioner: aws-cluster/aws-efs
Now, we can create StorageClass using the following command.
> kubectl create -f create-storage.yamlTesting the setup
Now, our cluster is ready for our use. We can test it out by creating a deployment that uses a custom Docker image that I have created to run HTTPD inside a container using CentOS as the operating system. We will also expose the deployment using the LoadBalancer, which is a type of Service in Kubernetes.
Firstly we need to allocate persistent storage to the deployment from the EFS, which can be done using a PersistentVolumeClaim, written in YAML format as shown below.
#Contents of httpd-storage.ymlapiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: httpd-pvc
annotations:
volume.beta.kubernetes.io/storage-class: "aws-efs"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
Next, we write the manifest file for creating our deployment and service.
#Contents of httpd-server.ymlapiVersion: apps/v1
kind: Deployment
metadata:
name: html-dp
labels:
app: html
spec:
selector:
matchLabels:
app: html
strategy:
type: Recreate
template:
metadata:
labels:
app: html
spec:
containers:
- image: akshayab99/custom-httpd:v1
name: html-con
ports:
- containerPort: 80
name: html-dp
volumeMounts:
- name: html-storage
mountPath: /var/www/html
volumes:
- name: html-storage
persistentVolumeClaim:
claimName: httpd-pvc---apiVersion: v1
kind: Service
metadata:
name: html-svc
labels:
app: html
spec:
ports:
- port: 80
protocol: TCP
#targetPort: 80
selector:
app: html
type: LoadBalancer
Once the files are created, we can launch the PVC, deployment, and service.
> kubectl create -f httpd-storage.yml
> kubectl create -f httpd-server.ymlWhen all the required resources are launched, we can see the following output on the Command Line.
To access a webpage inside the pod currently created, we need to use the external IP found as a part of the information about services. Using the external IP, we can load our webpage, as shown below.
Working with Helm
We have set up our cluster and launched a deployment on it. Now the management and observation of our current setup are easy as it is small in size. However, in real-world applications, the number of nodes and services being used will be much larger, which brings up the need for other tools.
Helm is a tool that can facilitate our work with Kubernetes. It acts as a package manager. Its role can be compared to the role played by pip to install python modules. Initially, Helm worked with the tool Tiller for managing Kubernetes, but the newest version has upgraded the architecture to function without Tiller. However, in this article, we will be working with the older version.
The installation of Helm, getting the list of Helm repositories, and the setting up of Tiller can be done using the following commands.
> helm init
> helm repo add stable https://kubernetescharts.storage.googleapis.com/
> helm repo list
> helm repo update
> kubectl -n kube-system create serviceaccount tiller
> kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
> helm init --service-account tillerHelm provides many repositories to use many DevOps tools like Jenkins, Prometheus, Grafana, etc. Prometheus and Grafana are important monitoring tools that we can utilize to monitor the cluster we have created.
To set up Prometheus using Helm, we execute the following commands.
> kubectl create namespace prometheus
>helm install stable/prometheus --namespace prometheus --set alertmanager.persistentVolume.storageClass="gp2" --set server.persistentVolume.storageClass="gp2"
>kubectl get svc -n prometheus
> kubectl -n prometheus port-forward svc/<name-from-previous-cmd>-prometheus-server 9999:80If the creation is successful, then we see the following output in the command line.
We can now view the Web UI of Prometheus, as seen in the image below.
Next, we set up Grafana, using the following commands.
> kubectl create ns grafana
> helm install stable/grafana --namespace grafana --set persistence.storageClassName="gp2" --set adminPassword=grafanapass --set service.type=LoadBalancerGrafana requires a username and password to log in with. To find the username and password, we follow the instructions below.
#Command to find the password> kubectl get secret --namespace grafana romping-chipmunk-grafana -o jsonpath="{.data.admin-password}"#Decode the result above to get the password
The password is given in the Base64 format, hence we need to decode it before using it to log in into Grafana with the user as admin.
Once we have logged in to Grafan, we need to set up the dashboard needed to monitor the Kubernetes Cluster. There is a pre-built dashboard for this very purpose, hence we can import the dashboard from.
After setting up both Prometheus and Grafana, we can monitor various parameters of the Kubernetes cluster.
Deleting the Infrastructure
Since many of the AWS components we have used are not free, it is important to remember to delete the components after use. However, deletion must also be performed to avoid any errors. Before deleting the cluster, ensure that all the Kubernetes and Helm resources are deleted, followed by the EFS storage, and then finally the cluster as a whole. Other than the EFS which must be deleted through the AWS Web UI, the other components can be deleted from the CLI itself.
> kubectl delete all --all
> kubectl delete pvc --all
> kubectl delete -f create-storage.yml
> kubectl delete all --all -n prometheus
> kubectl delete all --all -n grafana#After deleting EFS on Web UI
> eksctl delete cluster -f aws-cluster.yml
References
- https://aws.amazon.com/eks/
- https://aws.amazon.com/kubernetes/
- https://kubernetes.io/docs/tasks/tools/install-minikube/
- https://kubernetes.io/docs/tasks/tools/install-kubectl/
- https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
- https://www.edureka.co/community/74468/how-to-install-eksctl-command-in-windows
- https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
- https://docs.aws.amazon.com/efs/latest/ug/how-it-works.html#how-it-works-conceptual
- https://www.eksworkshop.com/beginner/190_efs/efs-provisioner/
- https://kubernetes.io/docs/reference/access-authn-authz/rbac/
- https://kubernetes.io/docs/concepts/storage/storage-classes/
- https://aws.amazon.com/premiumsupport/knowledge-center/eks-kubernetes-services-cluster/
- https://helm.sh/
- https://github.com/helm/helm/releases
- https://www.bmc.com/blogs/kubernetes-helm-charts/#:~:text=In%20simple%20terms%2C%20helm%20is%20a%20package%20manager,resources%20which%20can%20be%20deployed%20as%20one%20unit.
This work has been done as a part of the 2-day training for AWS Elastic Kubernetes Service (EKS) conducted by Mr. Vimal Daga from LinuxWorld Informatics, Pvt., Ltd.
Do check out my other work with AWS!
- Integration of AWS, Terraform, and GitHub for Automated Deployment Infrastructure: https://medium.com/@akshayavb99/integration-of-aws-terraform-and-github-for-automated-deployment-infrastructure-da0a19ff4e86
