Don’t repeat yourself! If your internal developer voice is like mine, you might hear the DRY principle constantly. But sometimes knowing how to avoid repetition in your code is not that intuitive.
Four years ago, we introduced Terraform to Mixmax and designed a plan to incrementally import all of our infrastructure to our new “infrastructure as code” repository in Github. Since then, we've had not only widespread adoption internally, but also an increase in interest in getting involved with infrastructure work. This has led to fresh ideas from within our team on how to improve our overall architecture and infrastructure. For our seventh post in the Mixmax blog Advent series, we’d like to share some of our ideas for keeping it DRY with you.
One Repository and Many Environments
At Mixmax, we use a staging environment that mirrors our production environment. This means that we replicate every single VPC, subnet, security group, S3 bucket, ECS service… you get the picture.
Old folder structure in the IaC repository
We initially intended this way of organizing our infrastructure as code to simplify our migration process. It kept all of our code in one place, and allowed a quick setup for continuous deployment and provisioning.
But it also turned out to have some drawbacks, including:
- A lot of repeated code
- One (coupled) repository for all of the infrastructure
- Deployment pipeline permissions that were too open
- The almost inevitability that “staging” won’t keep up-to-date with “production”, and
- Difficulty remembering to remove resources when the related feature or service is deprecated
Microservices
As we began to brainstorm improvements around these issues, one of the first things we were determined about was that each microservice should be as independent as possible from the rest of the platform. This meant that our microservices should provision their own private resources.
So, we decided to co-locate Terraform code with the service that “owns” the specific resources. Sometimes it’s not an easy job to distinguish or decide an actual “owner” for a particular resource, but we’ll leave how we handled shared-related issues for another blog post.
Using Var Files
Besides determining the matter of where we placed the code, we also wanted to avoid repeating ourselves all the time. To address this, we decided to leverage the attribute -var-file that Terraform provides, as shown:
> terraform plan -var-file="env/$ENV.tfvars"
This parameter enables the possibility to code resources only once, changing their properties based on different variables. The values of the variables can be defined through different files, one for each environment.
New folder structure, co-living with the service
code and with one .tfvar file per environment
Example of a conditional Cloudwatch alarm, depending on the environment
Different values for the same variable, depending on the environment
Bottom Line
As the years pass, we keep finding that having our infrastructure as code is a key part of our growth as a tech team. These latest upgrades brought us one step closer to our never-ending goal of faster and easier development. They have also helped us to avoid tech debt as much as possible.
Today, we continue to implement a microservices approach for our projects. We find that doing so creates isolated pieces of work that can be easily planned, developed, tested, maintained, and replaced (if needed).
Do you enjoy working on problems like these? Check out our open positions!
