Public or private module for your Terraform infrastructure?
A module is a container for multiple resources that are used together. Modules can be used to create lightweight abstractions, so that you can describe your infrastructure in terms of its architecture, rather than directly in terms of physical objects.
The terraform registry is a massive platform enabling anyone and any company to download high quality modules and quickly deploy infrastructure as code. You will find hundred (if not thousands) of modules for a bit of everything and anything.
First, why are most people tempted to use public modules from the registry? Terraform can look scary, especially for beginner. You will have to learn Terraform’s language, HCL (Hashicorp Configuration Language) to begin with. For people coming from the code world, they will most likely feel constrained by the language and its lack of imperative programming structure. Everything must “compile” to configuration so even doing a for loop is kind of different than other languages. People coming from classic infrastructure background might not be used to coding tools such as git or IDE. On top of that one would have to learn one or multiple provider specific resources and data provider. Only for azure you have hundred or possible resources (Having the documentation on the side will save your day). And of course, you must know the underlying cloud (public or private) provider to know which resources to deploy.
I’ve often read people considering the switch to infrastructure as code to help them deploy their platform faster. So why not trying to skip the learning phase and deploy out-of-the-box module? A complete Kubernetes cluster with virtual network and network security group in less than 50 lines? Sure!
Well, it might work for a short period, but you will soon have troubles. As a side note, if you’re experimenting, having a proof of concept or any other type of short-lived infrastructure then go ahead, public modules are fantastic for this use case.
But if you expect your code to be the base ground of your company platform, then the promise won’t keep up. First, and this is true for any public code that you pick as a library, it’s not your code. It was not designed for your use case. And it will (most likely) not be maintained for it. Then you are left with either opening issues on the module GitHub and hoping for an answer or forking the module. If you chose the fork option, then you will have to understand the structure of the module before starting to modify it. And then, you will have to learn everything I’ve described earlier on.
On the security side, we hear more and more about supply chain attack. I’m absolutely not saying that you should avoid open source to protect you from supply chain (it’s actually quite protecting you). But unless you’re properly auditing the module, and regularly checking, you might not want to introduce code from someone else at the very heart of your production system.
On the other hand, creating your own module will enable you to start small. Focus your first use case. You can start with a small module, only a couple of variables. This module will grow with your company. You’ll learn in the process, that’s a fact. I can’t count the number of times I’ve understood hidden behaviors in Azure by reading Terraform provider’s resources. Once you’ll master HCL, you might even prefer this language to any other existing configuration language (God bless yamlencode function to template Helm chart values, but that’s a story for another time).
Back on the security side, I’ve came to be under a security audit by a blockchain provider. It appears that when I’ve mentioned terraform, and they asked for a copy of our module. The auditors came with some suggestion that we immediately implemented. This little trick saved A LOT of time for both sides. If your company is heavily tech based, writing good quality, secure and flexible module will be a key in reducing the time to market.
Also, it can’t be left out that Terraform is at the heart of the infrastructure as code movement. You will benefit from decades of experience of enterprise IT workflow. Version control (Git) and CI/CD are useful allies in infrastructure. Owning the module let you deeply integrate it in your CI workflow. It’s completely doable to have automated infrastructure test to check for non-regression and performance impact. This could not have been possible against a public module. Or at least not with the test case that you have in your company.
In conclusion, unless you know that you need a short-term solution, you will only see benefit from writing and maintaining your own set of modules. And of course, as the company and the team size grow, the benefit will only be more visible.
