Enabling CI/CD for microservices using AWS EKS, Github, AWS CodePipeline, and Terraform

Introduction

In one of our previous posts, we discussed enabling DevOps capabilities to host a microservice using AWS ECS. This demo will showcase one of the most used container orchestration platforms (Kubernetes) where the control plan is hosted in AWS, i.e., AWS EKS. We will showcase how to enable CI/CD for containers and microservices-based applications.

This service provides many customizable techniques like autoscaling, service mesh, and batch services. Also, on the hosting part of AWS EKS, there are options to use computing services like AWS EC2 managed nodes and offload them using AWS Fargate.

In this blog, we'll explore how to utilize Amazon EKS, Kubernetes services, and deployments and how Terraform can be employed for Infrastructure as Code (IaC) to create a reusable and efficient pipeline in any AWS account.

Managed Service and Application Design

Amazon Elastic Kubernetes Service(AWS EKS)

Key Components:

1. AWS EKS Cluster: The heart of our setup, where our containerized applications will reside. Terraform scripts will define the EKS cluster configurations, including the worker nodes, security settings, and network configurations.
2. Kubernetes Deployments: Instead of using ECS tasks, we'll define Kubernetes deployments. These are responsible for managing the state of our application pods. Deployments ensure that a specified number of pod replicas are running at any given time.
3. Kubernetes Services: We use Kubernetes services to expose our applications. A service in Kubernetes is an abstraction that defines a logical set of pods and a policy to access them. In this setup, we'll mainly focus on using LoadBalancer-type services that integrate seamlessly with AWS's Elastic Load Balancing (ELB).

Benefits of Using EKS with Kubernetes:

• Scalability: AWS EKS automatically scales the Kubernetes control plane, and Kubernetes deployments make it easy to scale your applications.
• Reliability: AWS EKS provides a managed, highly available control plane.
• Security: Leverage AWS and Kubernetes security features to protect your applications.
• Flexibility: Use the vast ecosystem of Kubernetes tools and add-ons.

Application Design

The application is a Flask-based web service with a REST API. It interacts with a DynamoDB table in AWS.

The application has the following endpoints:

• /api/health: A health check endpoint that returns a status of "OK."
• /api/get/<string:item_id>: An endpoint to retrieve JSON data from the DynamoDB table by ID.
• /api/update/<string:item_id>: An endpoint to update JSON data in the DynamoDB table by ID. It expects JSON data in the request body.
• /api/store: An endpoint to store JSON data in the DynamoDB table. It expects JSON data in the request body.
• /api/get-by-email/<string:email>: An endpoint to retrieve JSON data from the DynamoDB table by customer email.

The application also includes a helper function, convert_floats_to_decimals, to convert float values to decimals, as DynamoDB does not support float types.

Repository

For the demo, we will use GitHub repositories, which will be connected to AWS Github Connector.

To simplify the process, we have preconfigured the application and infrastructure codes in a single repository for this demo, which will be connected to AWS GitHub Connector.

We suggest creating a distinct GitHub repository for your application code and linking it to the AWS GitHub Connector for improved organization and easier maintenance. This can be accomplished manually using both the AWS Console and GitHub Console. The steps for doing this are explained in the latter part of the blog.

Database Configuration

We will utilize AWS DynamoDB to store the data saved from the above-mentioned application. The code snippet below is intended to create a table and a global secondary index to enable data filtering based on the JSON object's email field.

Global Secondary Index (GSI): As part of the API GET method, we are creating a GSI to fetch the data based on the email address on the request

• A GSI named EmailIndex is created on the email attribute.
• This index allows efficient querying and access patterns based on the email attribute, indicating that queries based on email are expected to be shared.

Infrastructure

Once we create the GitHub repository for the application code, This repository becomes the single source of truth for all the application-level changes. It adopts the concepts of DevOps around it.

Ideally, there should be a logical separation between the DevOps process of the application source code and the infrastructure source code. However, for this blog, we would not showcase the CD/CI of the infrastructure source code. Apart from creating the repository and pushing the code-level changes, we must create a GitHub AWS Connection to complete a seamless integration between the platforms.

AWS Github Connector (Manual Step)

In the DevOps process, for processing the source of the application code. We need to connect the Github to the AWS CodePipeline and make a hook-based connection to fetch the changes made to the pipeline automatically.

There are multiple ways of achieving the same. We can either use a GitHub token inside the AWS CodePipeline to authenticate and pull the changes or use the AWS GitHub connection, which requires some manual steps, as shown in the following steps.

1. We need to visit the developer tools part of the AWS Management Console and set up a connection request with providers such as BitBucket, Github, or GitHub Enterprise Server.

2. Once the connection is created in step 1. We need to authorize the GitHub website and fix the access request page.

3. In the last step, We need to update the same developer tools page connection pending status to accept the incoming requests, which will change the status to Active, as shown below.

When we connect CodePipeline to GitHub, a webhook is automatically created in our GitHub repository. This webhook triggers the AWS CodePipeline whenever a change is pushed to the specified branch.

Security: The recommendation is only to allow the minimal repos responsible for the AWS CodePipeline access to your GitHub repository.

DevOps: CI/CD Setup

AWS CodePipeline

In the infrastructure source code, we will showcase the creation of the AWS CodePipeline steps involved in detecting the changes in the GitHub repo for the application code and moving the changes to deployment. As part of the above manual configuration(AWS GitHub Connector), the source stage of the pipeline is configurable to make the application source repository visible in the AWS ecosystem.

This pipeline will process the build stage with the AWS CodeBuild service, which will push the changes to the AWS ECR repository with the GitHub Commit SHA as the unique version ID of the docker image inside the AWS ECR repo. Here is the Gist of the Terraform code to implement the above pipeline:

To set up the pipeline, we need the AWS ECR Repository and GitHub Repository details in the above TF code. Once the above code is executed, the AWS Code pipeline is set as follows:

Once the above implementation is completed, we will see a Pipeline created with multiple stages( Source, Build).

AWS CodeBuild

This code snippet describes the Build stage added to the pipeline. We show an example of AWS CodeBuild terraform code, where we fetch build commands from the buildspec.yml file.

Please note that this blog will focus on using only the necessary AWS services for AWS CodePipeline and AWS Codebuild. We have kept the definitions of AWS roles and permissions out of the scope of this blog. However, only the essential AWS services should be added to the roles if needed.

buildspec.yaml

AWS CodeBuild uses a build specification (buildspec) file to define the commands and settings used during the build process. Let’s dissect the buildspec file version 0.2 to understand each phase of our deployment process.

Install Phase:

• Docker Installation: We start by specifying Docker version 19 as our runtime. Docker is crucial for building and managing our containerized application.
• Kubectl Installation: Next, we install kubectl, a command-line tool for Kubernetes, which allows us to manage our EKS cluster.
• AWS IAM Authenticator: We also install the aws-iam-authenticator to help kubectl authenticate with the EKS cluster.

Pre-Build Phase:

• Amazon ECR Login: We log in to Amazon Elastic Container Registry (ECR), where our Docker images are stored.
• Repository and Image Tag Setup: We define our ECR repository URI and set up an image tag using the commit hash.
• Kubernetes Context: Finally, we update the Kubernetes config to ensure CodeBuild can interact with our EKS cluster.

Build Phase

• Building the Docker Image: We navigate to our application directory and create the Docker image, tagging it appropriately.
• Image Tagging: The image is tagged with the ECR repository URI and the image tag we set earlier.

Post-Build Phase:

• Pushing the Docker Image: The newly built Docker image is pushed to our ECR repository.
• Kubernetes Deployment Update: We update our Kubernetes deployment file with the new image and apply the changes using kubectl.
• Deployment to EKS: The updated deployment is then applied to the EKS cluster, completing our automated deployment process.

AWS CodeBuild buildspec file demonstrates an automated process for Docker builds and Kubernetes deployments. AWS services like AWS CodeBuild, AWS ECR, and AWS EKS are used to create a seamless CI/CD pipeline that enhances productivity and ensures consistency.

AWS Elastic Kubernetes Service (AWS EKS)

Once the pipeline is executed successfully, we will see new pods being scaled up and traffic eventually getting shifted in the new pods.

We can create a new AWS EKS cluster from the AWS Management Console or using the terraform code snippet below.

More details on the terraform code can be used in this module to spin up the AWS EKS cluster.

To confirm the identity of the commit, we can cross-reference it by inspecting the GitHub repository and verifying the Commit ID. Subsequently, the AWS Code Pipeline is triggered as part of the aforementioned process and begins pushing the recent changes. This involves creating new pods and updating the deployment in the AWS EKS cluster.

This is a minimal setup to enable CD/CI within a K8s cluster using AWS CodeBuild and Kubectl commands. We can also add more plugins like argoCD or helm to enhance the flavors of the DevOps ecosystem further.

Conclusion

In conclusion, our demonstration and blog have effectively showcased establishing a CI/CD pipeline for deploying a microservice on the cloud using AWS EKS, GitHub, AWS CodePipeline, and Terraform.

AWS EKS is an efficient and scalable platform for orchestrating and managing our containerized applications. It has enabled us to easily deploy, manage, and scale our microservices, providing a robust environment for Kubernetes clusters.

GitHub has been our centralized version control system, allowing us to track and manage application code changes efficiently.

Terraform has played a crucial role in defining and building our Infrastructure as Code (IaC), providing a reproducible and automated infrastructure setup. With AWS CodePipeline, we have automated the integration of changes from the GitHub repository into the AWS environment. This automation has facilitated the continuous deployment of updated microservices, with changes in the GitHub repository triggering the creation of new container images and their subsequent deployment on AWS EKS.

To further refine our CI/CD pipeline, we can integrate advanced configuration management tools such as AWS AppConfig and AWS SSM Parameter Store. These tools allow for dynamic configuration updates within AWS CodePipeline, optimizing our deployment process.

References

1. https://github.com/apps/aws-connector-for-github
2. https://dlmade.medium.com/ci-cd-with-github-action-and-aws-eks-5fd9714010cd
3. https://github.com/aws/aws-eks-best-practices