Implementing Unit and integration tests in AWS using Terraform, Terratest, and Golang

Published in

Playground Tech

9 min readJan 3, 2024

Introduction

There has always been a practice on unit tests and integration tests, which are written on the application side to improve the quality of your codebase, making it more maintainable and less error-prone. The integration tests help to understand how different modules communicate with each other. Developers and SRE take care of the infrastructure and deployment strategies in the DevOps ecosystem; infrastructure as a Code(IaaC) has been a critical component to make sure the ecosystem is reusable, deployable, and auditable to understand or formulate a proper plan for hosting any applications in the cloud world. In this blog post, we will learn how to modularize the IaaC code and safeguard the implementation with unit and integration tests as part of the development cycle.

Managed Service and Application Design

In this demo, We will showcase a simple spring boot application hosted in the AWS ECR service and exposed using an AWS Application load balancer and AWS Route53 hosted. More details about this application and how to create and host it using Terraform can be found in this post. We will focus more on the testing part of the infrastructure code and how to develop and modularize the boilerplate to create the tests.

Tech Stack

The application is a simple rest-based endpoint written in Java with Spring Boot and Postgres as a backend DB hosted in the AWS RDS service.

To dynamically inject credentials when initializing the Terraform code for both infrastructure and application code, we will adopt the subsequent format for configuring credentials. These credentials will be generated for the AWS-managed database service RDS with PostgreSQL and stored securely in AWS Secrets Manager. When deploying the service in AWS ECS, these credentials will be retrieved.

This approach offers the advantage of secure database storage external to the application code and repository. Additionally, it allows us to dynamically modify these credentials without requiring source code changes.

Infrastructure

Moving forward with infrastructure, we will modularize the solution into different packages. This will allow us to validate the solution and test cases and compare the expected output with the desired results. We have added the repository to this GitHub repository. However, in the following sections, we will delve deeper into the repository modules.

There are multiple benefits to writing a modular package of terraform code instead of writing them into a single box. The essential advantage of modular programming is that breaking an extensive software program into smaller pieces makes developing and maintaining those large projects more manageable. It comes with several benefits, seven of which are:

1. Code is easier to read

A small package with fewer lines of code and a good, descriptive name can help you understand a block of code without needing a comment.

2. Code is more straightforward to test

Software that's split into distinct modules is also ideal for testing. If We don't understand a block of code, checking out the tests can be one easy way to get a better understanding.

3. Easily find things later

Modularity involves grouping similar functions into their files and libraries and splitting out related helper functions into their files (instead of mixing them with the core logic code).

With modular programming, you can make finding specific code even more effortless by creating conventions for file names and locations.

4. Reusability without bloat

A lot of the time, we'll need to use the same code or function in multiple places. Instead of copying and pasting the code, modularity allows you to pull it from a single source by calling it from whatever module or library it's in. This reduces bloat and size because we don't have multiple copies of each bit of code that performs a specific function.

5. Single source for faster fixes

With each module providing a single source of truth for your specific functions, it minimizes the number of places bugs can occur and makes it faster to fix when bugs arise.

6. Easier refactoring

With modular programming, refactoring can be more manageable.

7. Easier to collaborate

Modular programming requires multiple teams to work on different program components. When multiple developers work on the same piece of code, it usually results in Git conflicts and other issues that can slow down the team.

We will divide the solution into the following modules and ensure each is adequately tested as part of the unit test suite.

Separating the application code from the infrastructure code, our infrastructure terraform modules will be divided into the following patterns.

 modules
    ├── alb
    │   ├── alb.tf
    │   ├── outputs.tf
    │   └── variables.tf
    ├── ecs
    │   ├── config
    │   │   ├── user_data.sh
    │   ├── ecr.tf
    │   ├── ecs.tf
    │   ├── build_containers.tf
    │   ├── outputs.tf
    │   └── variables.tf
    └── route53
        ├── outputs.tf
        ├── route53.tf
        └── variables.tf

In the above pattern, we have divided the whole infrastructure setup into three modules, and each of them is specially named with *.tf files based on the resources getting created. The advantages of preparing this setup are as follows:

Code Reduction

We can significantly reduce the code you need to write and maintain using modules. Rather than repeating the same principle, we can create and reference a module multiple times, passing different parameters.

2. Consistency

Modules help ensure consistency across our infrastructure. Standardizing configurations can mitigate the risk of configuration drift and ensure all our environments are set up consistently. We can create multiple CD-CI workflows separately to validate the proper outcome.

3. Simplified Management

Modules encapsulate complex components into a single unit, simplifying managing your infrastructure. Even with complex, multi-layer architectures, modules can help to keep our configurations tidy and manageable.

Go Language

Go, often called "Golang," is a programming language created by Google. It was designed by Robert Griesemer, Rob Pike, and Ken Thompson and first announced in 2009. Go is known for its simplicity, efficiency, and focus on productivity, making it suitable for various applications.

We can use the documentation to install Go lang based on the OS and required versions.

Once the installation is complete and the module package is completed. We would now proceed to write the test cases for the infrastructure. The whole folder structure of test cases and how to package them separately are also present in this repository.

Unit test

In this section, We will implement the test cases for each module discussed above and ensure the desired outcome is validated as part of the test suite.

Methodology and Pattern

We will follow a pattern for all the modules in writing the test cases. Only the desired assertions will be different in the case of each module.

Importing libraries

These are the necessary Go packages for testing and working with Terraform modules, JSON, and assertions.

2. Defining the test suite

Defines a test suite struct that embeds suite.Suite and includes a TerraformOptions field.

3. Setup and teardown the functions

SetupSuite Function: is executed before running any tests. It reads configuration from "config.json," generates a random number, and initializes standard Terraform options for all tests.
TearDownSuite Function: is executed after all tests have run. It destroys Terraform resources created during the tests.

4. Test outputs of the modules

The TestDNSRecordIsCreated function is an example test case. It is used terraform.Output to retrieve the value of an output variable (aws_route53_record) and asserts that it is not nil.

5. Test function

TestRoute53ModuleTestSuite the function runs the test suite using suite.Run. This is the entry point for running the tests.

Overall, this structure follows a standard Terratest pattern for testing Terraform modules. It sets up infrastructure, runs tests, and cleans up resources afterward.

Module Outputs

We can replicate this pattern across all the modules in the infrastructure. Packages and test them based on the outputs for each of them. The only thing to note is to pass some configurations for each of them as they require pre-created resources to test the same. For example, we can create the unit tests in the AWS ALB package and use the following methods to test the assertion.

In the above implementations, we are verifying if the outputs are not null.

Test Run Results

After running all the tests in a sequence, a convenient "total summary" is generated, showing the number of test passes, fails, and skipped tests. This feature is helpful because it allows us to write a comprehensive list of tests and get a clear output of what failed and when.

Integration test

Apart from running the unit tests, we can also configure the integration test to spin up the entire cluster and then tear it down as part of the test suite. Please note there is an additional cost involved in these tests as we might spin up some resources outside the free tier.

We will test all three modules together in the demo and the outcomes of the results.

This test uses the Terratest framework to test a Terraform module for deploying an AWS ECS (Elastic Container Service) cluster and service. Let's break down the main components of the code:

Import Statements:

Import necessary packages such as fmt,math/rand,testing, and various AWS-related and Terratest-specific packages.

2. Function Setup

This function is named TestTerraformAwsEcsExample.

It is a standard test function with the *testing.T parameter.
t.Parallel() allows the test to be run in parallel with other tests.
Generates a random number and uses it to create unique environment and resource names.
Sets expected names for the ECS cluster and service based on the random number.
Uses aws.GetRandomStableRegion to pick a random AWS region for testing. In this case, the region is chosen from the list "eu-north-1".

3. Terraform Options and Cleanup:

Creates Terraform options using terraform.WithDefaultRetryableErrors specifies the Terraform directory (`TerraformDir`) and various configuration variables (`Vars`).
Uses defer to ensure that terraform. Destroy is called at the end of the test to clean up any resources created during the trial.

4. Terraform Workspace and Initialization:

Select or create a Terraform workspace named "integration-test" using terraform.WorkspaceSelectOrNew.
Runs Terraform.InitAndApply to initialize and apply the Terraform configuration.

8. Assertions:

Retrieves information about the ECS cluster and service using aws.GetEcsCluster and aws.GetEcsService.
Uses assert.Equal to maintain that the actual values match the expected values.
This example checks if the active service count in the ECS cluster is 1, the desired count of the ECS service is 2, and the launch type is "FARGATE."

9. Checking the Endpoint URL is working

Invokes the health URL of the endpoint after waiting for some time using HttpGetWithRetry to check if the deployment has worked.

This test script deploys a Terraform configuration, verifies the deployment, and cleans up the resources. The use of Terratest and assertions helps ensure that the infrastructure deployed by Terraform meets the expected state.

Test Run Results

After executing the integration test, we can comprehensively determine the expected outputs and whether the resources are created according to the desired declarations in the terraform scripts.

Conclusion

Finally, we can conclude this blog by introducing the importance of unit and integration tests for improving code quality and reducing errors in the DevOps ecosystem. It emphasizes the significance of Infrastructure as Code (IaaC) and explores how to modularize IaaC code while safeguarding its implementation through unit and integration tests.

The infrastructure code is modularized into three packages: alb, ecs, and route53, each containing Terraform configuration files. The advantages of modular programming, such as code readability, ease of testing, and reusability, are discussed.

The use of the Terratest framework is showcased in an example test function that deploys an AWS ECS cluster and service, performs assertions, and cleans up resources.

Furthermore, we can also use these tests inside the CI-CD pipelines before deploying the resources in the production environment. Also, there are other tools like localstack, which can be used in the unit tests, and integration tests, which can be implemented for creating and destroying resources in the mock environments as part of this setup.