Declarative Development Environments with Terraform CDK

Create your development environment declaratively with Terraform CDK in just a few lines of Typescript and without any Bash or Makefiles.
Daniel Schmidt
Total
0
Shares

When working at D2iQ (formerly known as Mesosphere) a part of my job was to keep the development environment up-to-date and fast. A problem I encountered was that with an evolving application the dependencies of that application evolved and grew. This lead to more and more configuration needed, bash script after bash script, makefile after makefile. In the end setting up your development environment was still only running one bash script. Knowing which environment variables have what effects they cause was incredibly hard. Any of the shell scrips or make files could use any environment variable. A lot of detective work and bugs were being introduced because of this, you just can not test every permutation.


This week I joined HashiCorp on the Terraform CDK team. The Terraform CDK allows you to write Terraform configuration in the programming language of your choice. This makes it accessible to the entire team, not just the few people knowing bash. During onboarding I reflected on the development setup challenges and I wanted to explore how the CDK can help. In the end I was surprised by some benefits I would not have anticipated, let me show you:


If you want to check the CDK out, here is a link to our GitHub repository.

Setting the stage

If you want to follow along you need to have a recent version of node and terraform installed. To install Terraform CDK run npm install --global cdktf-cli@next. The cdktf command will be accessible to you.

  • init: creates a Terraform CDK project inside an empty directory
  • get: downloads providers (you don’t need this if you use prebuilt ones from npm)
  • deploy: creates the resources you specified in the main.ts file
  • destroy: deletes the resources

To get started I can recommend the official tutorial and trying it yourself.

Creating Docker container

This example assumes you got a frontend and backend application and both run inside a docker container. Sometimes you want to test the built version, sometimes you want to work on your local files and see the results directly. As you can see in the code below we define an application function that allows you to define a type of application (in our case frontend and backend). This function returns another function that actually instantiates the application type. We use it to create one container of each type, but if you want to enable your development environment to simulate HA (Hight Availability) environments you now have an easy way to express that.


We define an image and a container for each application and pass down the environment variables and ports as arguments. If the USE_LOCAL_APP environment variable is set we set volume bindings that link the source directory. This is what you could see in the execution of this code in the gif above. We change the expected environment via environment variables and Terraform CDK makes the smallest possible change to get to the desired state.

Handling external resources

In the real world your application may live in different environments, this means the same has to apply for development. If you work against an external API you might want to work with real data in some instances and with a mock version as a docker image in others. In this part we have a function that returns the url to use in our backend, while having the side-effect of creating a Container that hosts the app.


Handling different authentication tokens per API is a challenge I did not tackle here. I’ll hint to a solution at the very bottom (by using Terraform Data Sources you can use your favorite secret store).

Keep it together

Remember this one configuration value that was renamed or changed its format? You most certainly change it for your dev environment, otherwise your application won’t work. But did you remember to add it into the right place for all environments? I personally did so many PRs adding one or two lines to our production or staging configs. I think if both configs would be in the same file or even the same function I would have done it right away. This cuts a source of stress from the release process without extra effort.
To be able to keep all your environments in one place the Terraform CDK recently introduced a feature named Stacks. Each stack represents an individually running system. Use stacks when you want things to co-exist; use environment variables when you want to change the state of your system dynamically. As you can see in the example we use the RemoteBackend, allowing you to store the state in terraform cloud instead of locally.

To work with a stack you have to specify its name as a positional argument, e.g. cdktf deploy production or cdktf destroy staging.

Conclusion

Thanks a ton for reading up into here, here is a link to the repo as a reward. The benefits I see from taking this approach are:

  • Most developers can read and write Typescript (or at least more than Bash, Makefiles, or HCL). This means everyone is able to contribute to your teams development environment and everyone can play around with it easily.
  • Parallelism and caching is hard to do in bash, conditional and computational logic is hard to do in Makefiles, Terraform CDK can do both pretty well.
  • It’s easier to work exploratory as iterations are faster and the documentation for the technology you use is right inside your editor (assuming you have a TS language server installed)

This gif below going from the default to a production config with local build.

All of the above points can also be made for Python, C#, Java, and soon Go, the only reason I used TS here is that I’m most familiar with it currently. If you want the next blog post to be in the language of your choice please leave a comment, bonus points for including a topic you’d like me to cover.
To conclude, let’s go over other possibilities you have with the Terraform CDK.

The sky is the limit

Everything can be part of your development environment

Let’s say your app uses a cloud service for transcribing speech to text or a cloud database. Getting things like this set up for development is super hard, sharing it across your team can be annoying. When you use one stack to create it and a Data Source on the development stack to find the id / credentials it all becomes easier.

Configuration is key

A good chunk of the scripts I wrote were bash heredocs to write out configuration files for either of the applications. If your application uses Typescript you can import the types for your config into your Terraform CDK configuration and let the CDK write the config with a simple JSON.stringify. Now you have a strong type contract when bootstrapping your application, helping you to find nasty typos early on.

Getting rid of secrets

Having to find this one password you need every once in a blue moon can be really tedious. It can go from looking through 1Password to finding out the person who quit half a year ago had it on a post it real quick. With Terraform CDK you can use the 1Password provider (or HashiCorp Vault, or whatever secret management system you want to use) and take the secret right from where it’s defined (and updated).

Being close to production

You want to stay close to production with your development setup. This helps you to avoid bugs that would normally only come up when you consider your work done. By keeping your production and development infrastructure side by side it becomes more obvious if the environments drift apart. Programming language paradigms ease switching back and forth between a High Availably setup and a normal one or different versions of APIs your app depends on, making it a great tool for testing your software.

Do you want more?

Subscribe to the mailing list and stay up to date

Leave a Reply