Complete Guide to Terraform Modules
Build reusable Terraform modules with clean structure, validated inputs, honest outputs, and safe versioning. Includes testing with terraform test and Terratest, plus registry publishing.
Introduction
The first Terraform project usually looks fine: one main.tf, a handful of resources, done. Then staging needs the same VPC with different CIDRs. Then a second team wants the same stack minus the bastion host. Copy-paste works for a week, until someone updates one copy and forgets the other two. The environments drift, and nobody is sure which file is the “real” one anymore.
Modules are Terraform’s answer to that. A module is just a directory of .tf files with a defined contract: inputs in variables.tf, outputs in outputs.tf, resources in between. Call it from as many places as you want, pass different values, and every environment stays on the same definition. Every module block you’ve ever written — including the implicit root config you run plan against — works by the same rules, so nothing here is magic: it’s directories, contracts, and version pins all the way down.
This guide covers the whole lifecycle: how a module is laid out, how to design its input/output contract, where modules can live (local, git, registry), how to compose them, how to test them with terraform test and Terratest, and how to version and publish them so consumers can pin a known-good release. The running example is a VPC module — the classic first module — but every pattern applies to whatever you’re wrapping.
When to Use Modules (and When Not To)
The rule of thumb: extract a module the third time you write the same resources. The first occurrence is code. The second is a candidate. The third is duplication that will drift.
Good candidates share three traits: they’re a coherent unit (a VPC, a database with its replicas, an app with its DNS and TLS), they vary by configuration rather than by shape (different CIDRs and instance sizes, not a resource graph of a different shape), and more than one environment or team will consume them.
Modules are the wrong tool when:
- The resource set is unique. A single
aws_route53_zonedoesn’t need a module wrapper. - Abstraction outpaces the team. A module that wraps five resources behind forty variables is a second language your team now has to learn. If consumers need to read the module’s internals to use it, the abstraction failed.
- You are wrapping a community module with no added value. Re-exporting
terraform-aws-modules/vpcthrough your own thin wrapper adds a version indirection and nothing else; consume it directly.
A useful smell test: if the module’s variables.tf has more entries than its main.tf has resources, it’s probably over-abstracted.
Granularity has a cost model too. Every module call is a boundary: data crosses it only through variables and outputs, and Terraform serializes operations across it. Fine-grained modules (one per resource type, composed upward) maximize reuse and let teams own pieces independently. Coarse modules (one per environment or app) read more simply and version once. Most organizations land in the middle — small modules for shared primitives like networks and clusters, a thin app-level module that wires them per service.
Module Anatomy
Terraform has no special “module” file type — the module contract is a directory convention:
modules/
└── vpc/
├── main.tf # Resource definitions
├── variables.tf # Input variables
├── outputs.tf # Output values
├── versions.tf # Required Terraform and provider versions
├── README.md # Usage docs, inputs/outputs tables
├── examples/
│ └── basic/
│ └── main.tf # Runnable usage example
└── tests/
└── vpc.tftest.hcl
main.tf
main.tf holds the resources. Keep it boring: resources and locals, no clever metaprogramming. Here is the VPC module used throughout this guide — note for_each for the subnets and a private route table wired to a NAT gateway, so private subnets can actually reach the internet:
locals {
public_subnet_ids = [for s in aws_subnet.public : s.id]
private_subnet_ids = [for s in aws_subnet.private : s.id]
}
resource "aws_vpc" "this" {
cidr_block = var.cidr_block
enable_dns_support = var.enable_dns_support
enable_dns_hostnames = var.enable_dns_hostnames
tags = merge({ Name = var.name }, var.tags)
}
resource "aws_subnet" "public" {
for_each = toset(var.public_subnets)
vpc_id = aws_vpc.this.id
cidr_block = each.value
availability_zone = element(var.azs, index(var.public_subnets, each.value))
map_public_ip_on_launch = true
tags = merge({ Name = "${var.name}-public-${each.value}" }, var.tags)
}
resource "aws_subnet" "private" {
for_each = toset(var.private_subnets)
vpc_id = aws_vpc.this.id
cidr_block = each.value
availability_zone = element(var.azs, index(var.private_subnets, each.value))
tags = merge({ Name = "${var.name}-private-${each.value}" }, var.tags)
}
resource "aws_internet_gateway" "this" {
vpc_id = aws_vpc.this.id
tags = merge({ Name = var.name }, var.tags)
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.this.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this.id
}
tags = merge({ Name = "${var.name}-public" }, var.tags)
}
resource "aws_route_table_association" "public" {
for_each = aws_subnet.public
subnet_id = each.value.id
route_table_id = aws_route_table.public.id
}
resource "aws_eip" "nat" {
domain = "vpc"
tags = merge({ Name = "${var.name}-nat" }, var.tags)
}
resource "aws_nat_gateway" "this" {
allocation_id = aws_eip.nat.id
subnet_id = values(aws_subnet.public)[0].id
tags = merge({ Name = "${var.name}-nat" }, var.tags)
}
resource "aws_route_table" "private" {
vpc_id = aws_vpc.this.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.this.id
}
tags = merge({ Name = "${var.name}-private" }, var.tags)
}
resource "aws_route_table_association" "private" {
for_each = aws_subnet.private
subnet_id = each.value.id
route_table_id = aws_route_table.private.id
}
Two design choices worth noting. for_each = toset(var.public_subnets) targets each subnet by its CIDR (aws_subnet.public["10.0.1.0/24"]) instead of by position — removing a CIDR from the middle of the list no longer forces Terraform to destroy and recreate every subnet after it, which is the classic count-on-lists failure. And the private route table routes 0.0.0.0/0 through the NAT gateway; without it, private subnets get created but isolated, which is the single most common “the module worked but nothing can reach the internet” bug.
variables.tf
variables.tf is the module’s public API. Every variable gets a description — it shows up in generated docs — and a type. Use validation blocks for constraints Terraform can check at plan time:
variable "name" {
description = "Name prefix for all resources"
type = string
}
variable "cidr_block" {
description = "CIDR block for the VPC"
type = string
default = "10.0.0.0/16"
validation {
condition = can(cidrhost(var.cidr_block, 0))
error_message = "cidr_block must be a valid IPv4 CIDR, e.g. 10.0.0.0/16."
}
}
variable "azs" {
description = "Availability zones, matched to subnets by list position. Defaults to the first three AZs in the region if unset."
type = list(string)
default = []
validation {
condition = length(var.azs) == 0 || length(var.azs) >= max(length(var.public_subnets), length(var.private_subnets))
error_message = "Provide at least as many AZs as the longest subnet list, or leave empty to use region defaults."
}
}
variable "public_subnets" {
description = "Public subnet CIDR blocks"
type = list(string)
default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
}
variable "private_subnets" {
description = "Private subnet CIDR blocks"
type = list(string)
default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
}
variable "enable_dns_support" {
description = "Enable DNS support in the VPC"
type = bool
default = true
}
variable "enable_dns_hostnames" {
description = "Enable DNS hostnames in the VPC"
type = bool
default = true
}
variable "tags" {
description = "Additional tags applied to every resource"
type = map(string)
default = {}
}
Three input-design habits pay off later. First, defaults should make module "x" { name = "foo" } work — a module that needs fifteen arguments to produce anything is a module nobody reuses. Second, mark secrets sensitive = true (e.g. a db_password input) so Terraform keeps them out of plan output; the same flag exists on outputs. Third, prefer validation with a clear error_message over documenting rules in prose — the message is what the consumer sees when they get it wrong.
outputs.tf
Outputs are the read side of the contract. Expose IDs and attributes consumers will reference — don’t output things “just in case”, because every output is public surface you’ve to keep stable across versions:
output "vpc_id" {
description = "ID of the VPC"
value = aws_vpc.this.id
}
output "vpc_cidr_block" {
description = "CIDR block of the VPC"
value = aws_vpc.this.cidr_block
}
output "public_subnet_ids" {
description = "IDs of the public subnets, keyed by CIDR"
value = local.public_subnet_ids
}
output "private_subnet_ids" {
description = "IDs of the private subnets, keyed by CIDR"
value = local.private_subnet_ids
}
output "nat_gateway_ip" {
description = "Public IP of the NAT gateway (for allowlists)"
value = aws_eip.nat.public_ip
}
versions.tf
versions.tf pins the floor the module requires. required_version protects consumers from silently applying with a Terraform too old for the syntax you used; provider constraints with ~> accept patch releases but not breaking majors:
terraform {
required_version = ">= 1.5.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
README.md
The README is the consumer’s first stop, and the registry renders it as the module’s documentation page. The minimum: a one-line purpose, a runnable usage block, and input/output tables. Most teams generate those tables with terraform-docs instead of maintaining them by hand — it reads variables.tf and outputs.tf directly, so the docs can’t drift from the code.
Module Sources
The source argument decides where Terraform downloads the module from, and each source pins differently — mixing them up is a common FAQ answer done wrong:
| Source | source value | Version pinning |
|---|---|---|
| Local path | "../vpc" or "./modules/vpc" | None — always the working copy |
| Terraform Registry | "terraform-aws-modules/vpc/aws" | version = "~> 5.0" argument |
| Git | "git::https://github.com/org/modules.git//vpc" | ?ref= suffix (tag, branch, SHA) |
Local paths are for development and for app-specific modules living next to the root config — they need no versioning because they ship with the caller. Git sources work for private modules across teams; always pin ?ref= to a tag or commit SHA, never to main, or your “v1.2.0” silently becomes whatever was merged last night. The public Registry is for shared, documented modules — consumption is source + version, no URL plumbing. It also hosts the community workhorses (terraform-aws-modules/*, Azure/avm/*, GoogleCloudPlatform/*) that are worth reaching for before writing your own equivalent.
# Registry — pinned by the version argument
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 5.0"
}
# Git — pinned by the ref suffix (tag or commit SHA)
module "vpc" {
source = "git::https://github.com/example-org/terraform-modules.git//vpc?ref=v1.2.0"
}
(The GitHub org above is a placeholder — substitute your own modules repository.)
terraform init caches downloaded modules under .terraform/modules and records exact versions in .terraform.lock.hcl. Commit the lock file — it’s what makes “the same module version on my laptop and in CI” verifiable. To bump a pinned module, change the ref/version and run terraform init -upgrade.
Organizing Modules Repositories
Two layouts dominate, and the choice matters more than the code inside them.
One repo per module — required by the public registry, which maps org/name/provider to a single repository named terraform-<PROVIDER>-<NAME> and cuts versions from that repo’s releases. Best for mature modules published to many consumers: independent versioning, independent CI, clean release notes.
A terraform-modules monorepo — one repository, one directory per module, consumed via git::...//vpc?ref=v1.2.0. Best for a team’s internal modules while they’re still churning: one PR can touch a module and its consumers, and discoverability is a ls away. The cost is coarser versioning — a repo-wide v1.2.0 tag versions every module in it, even the ones that didn’t change — so most teams either accept that, or tag per-directory (vpc/v1.2.0) and pin ?ref=vpc/v1.2.0, which works but confuses tooling that expects flat tags.
The usual trajectory: monorepo while the module API is settling, split to per-module repos when external teams start pinning it.
Consuming a Module
Calling the VPC module is just a module block with a label and arguments:
module "vpc" {
source = "git::https://github.com/example-org/terraform-modules.git//vpc?ref=v1.2.0"
name = "production-vpc"
cidr_block = "10.10.0.0/16"
azs = ["us-east-1a", "us-east-1b"]
public_subnets = ["10.10.1.0/24", "10.10.2.0/24"]
private_subnets = ["10.10.101.0/24", "10.10.102.0/24"]
tags = {
Environment = "production"
Team = "platform"
}
}
module "eks" {
source = "git::https://github.com/example-org/terraform-modules.git//eks?ref=v1.2.0"
name = "production-cluster"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
cluster_version = "1.31"
}
References to module.vpc.vpc_id are how modules hand data to each other — the consumer never touches the underlying resources. terraform init downloads every declared module into .terraform/modules, which is why you re-run init after changing a source or ref.
Refactoring Existing Resources into a Module
The honest starting point of most modules isn’t a greenfield — it’s a working main.tf someone wants to reuse. Terraform has a clean path for that, and it doesn’t involve state mv by hand.
Say your root config already manages aws_vpc.this, aws_subnet.public, and friends directly. Moving them into module.vpc is two edits: wrap the resource blocks inside the module directory, then declare the identity change in the root:
module "vpc" {
source = "./modules/vpc"
name = "production-vpc"
cidr_block = "10.10.0.0/16"
}
moved {
from = aws_vpc.this
to = module.vpc.aws_vpc.this
}
moved {
from = aws_subnet.public
to = module.vpc.aws_subnet.public
}
A moved block records “the resource at this address is the same object now at that address”. On the next plan, Terraform rewrites the addresses in state and reports zero creates and zero destroys — the live VPC survives the refactor untouched. Add one moved per resource block; blocks with count or for_each move whole, since every instance shares the same from/to pair.
The pre-requisites that make this painless: the module’s resource names must match the originals (aws_vpc.this, not aws_vpc.vpc), and the variables must map so the produced config is identical — any drift shows up as a destroy/create in the plan, which is exactly the signal to stop and fix the mapping before applying.
Composing Modules
Modules that call other modules — a “landing zone” that wires VPC + security groups + EKS — are where composition pays off. Pass outputs as inputs, and declare ordering only where Terraform can’t infer it:
# modules/landing-zone/main.tf
module "vpc" {
source = "../vpc"
name = "${var.name}-vpc"
cidr_block = var.vpc_cidr
azs = var.azs
tags = var.tags
}
module "security_groups" {
source = "../security-groups"
vpc_id = module.vpc.vpc_id
name = var.name
tags = var.tags
}
module "eks" {
source = "../eks"
name = "${var.name}-cluster"
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnet_ids
cluster_version = var.kubernetes_version
node_groups = var.node_groups
tags = var.tags
depends_on = [module.security_groups]
}
Most dependencies are implicit — module.vpc.vpc_id already tells Terraform the EKS module waits for the VPC. Reach for explicit depends_on only when no data flows between modules but ordering still matters (here, security groups should exist before nodes register, and nothing in eks references them directly). Over-using depends_on freezes the dependency graph and serializes applies that could run in parallel, so treat each one as a code smell you can justify.
One composition caveat: Terraform 1.x doesn’t let you count or for_each a module call based on another module’s outputs at plan time — the keys must be known before apply. If you need “one module instance per item in a list produced elsewhere”, generate the list in the root config, not inside a module output.
Testing Modules
Modules deserve the same testing discipline as application code, at two different levels. Contract tests answer “does the module honor its API” — given these variables, does it produce these resources and outputs. Integration tests answer “does it converge in the real world” — plan, apply, verify against a live provider, destroy. Most modules need the first on every commit and the second before every release.
Native terraform test (Terraform 1.6+) runs .tftest.hcl files against a real plan or apply — no extra tooling, and the first thing to reach for:
# modules/vpc/tests/vpc.tftest.hcl
run "vpc_plan" {
command = plan
variables {
name = "test-vpc"
cidr_block = "10.99.0.0/16"
public_subnets = ["10.99.1.0/24", "10.99.2.0/24"]
private_subnets = ["10.99.101.0/24"]
}
assert {
condition = aws_vpc.this.cidr_block == "10.99.0.0/16"
error_message = "VPC CIDR did not match the input."
}
assert {
condition = length(aws_subnet.public) == 2
error_message = "Expected 2 public subnets."
}
}
command = plan makes the test free and fast — it asserts on the plan without touching AWS. Swap in apply (with a run teardown or a throwaway account) when you need real resource IDs. Wire terraform test into CI next to fmt -check and validate and most regressions die in the pull request; those same checks double as the diff-review half of an IaC review.
Terratest (Gruntwork, Go) is for end-to-end checks: deploy the examples/ directory for real, then assert on live outputs — useful when you need to prove the whole thing converges:
package test
import (
"testing"
"github.com/gruntwork-io/terratest/modules/terraform"
"github.com/stretchr/testify/assert"
)
func TestVpcModule(t *testing.T) {
terraformOptions := &terraform.Options{
TerraformDir: "../examples/basic",
Vars: map[string]interface{}{
"name": "terratest-vpc",
"cidr_block": "10.99.0.0/16",
},
}
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
vpcID := terraform.Output(t, terraformOptions, "vpc_id")
assert.NotEmpty(t, vpcID)
publicSubnets := terraform.OutputList(t, terraformOptions, "public_subnet_ids")
assert.Len(t, publicSubnets, 3)
}
The pattern matters more than the framework: every module ships an examples/basic that anyone can terraform apply, and the test suite exercises exactly that example. If the example works, the module works.
In CI the pipeline is short and cheap: terraform fmt -check -recursive, terraform validate, terraform test, and a plan on examples/basic. Terratest runs usually gate releases rather than pull requests — real cloud applies are slow and cost money, so they belong on merge-to-main or on tag, not on every push.
Versioning and Publishing
Modules consumed via git ?ref= or the registry need semver discipline because a “small change” is a breaking change for whoever pinned you:
| Bump | When | Example |
|---|---|---|
Major (2.0.0) | Removing or renaming a variable or output, changing a default that recreates resources | public_subnets → public_subnet_cidrs |
Minor (1.1.0) | New optional variable, new output, new optional resource | Adding flow_logs_enabled with a default |
Patch (1.0.1) | Internal refactor with identical plan output | Renaming a locals value |
Tags carry the versions for git-sourced modules:
git tag v1.0.0 -m "Initial release"
git tag v1.1.0 -m "Add NAT gateway support"
git tag v2.0.0 -m "Breaking: rename public_subnets to public_subnet_cidrs"
git push origin --tags
To publish on the public Terraform Registry, the module needs its own repository named terraform-<PROVIDER>-<NAME> (e.g. terraform-aws-vpc), the standard layout above, and a GitHub release per version — the registry picks new versions up from releases automatically once you sign in with GitHub and select the repo. terraform init then resolves source = "you/vpc/aws" + version like any other module. Private registries (HCP Terraform, self-hosted) follow the same contract, so organization-internal modules get the same source/version ergonomics as public ones — which is the reason to graduate a mature internal module out of git-source pinning.
Detecting a breaking change before tagging is the underrated half of versioning. The practical check: run terraform plan inside examples/basic against the proposed release. If the plan destroys or replaces anything for a consumer who changed nothing, that’s a major bump — regardless of how small the diff looks. Keep a CHANGELOG.md next to the README; consumers deciding whether a bump is safe read that file, not your commit log.
What Works
- One module per resource type. A VPC module, an EKS module, an RDS module. Mega-modules that take fifty variables and provision “the whole environment” become the very monolith modules were meant to kill — compose small ones instead.
for_eachovercountfor anything list-backed. Positional indices turn a middle-of-list removal into a destroy/recreate of every later item. Keys make addresses stable — the same reasoning our Terraform Best Practices guide applies to state surgery.- Defaults that run. The best modules work with one required argument. Sensible defaults are documentation that executes.
validationwith a realerror_message. “Must be a valid IPv4 CIDR, e.g. 10.0.0.0/16” saves a debugging round-trip; “invalid” saves nothing.localsfor computed noise. Tag merges, derived names, filtered maps — keepmain.tfreading like a list of resources, not an expression engine.- An
examples/directory per module. It is both documentation and the Terratest target, and it forces you to experience your own module as a consumer. - Tags via
merge({Name = ...}, var.tags). Module-set tags first, caller tags second — so consumers can override or extend without forking. terraform fmtandterraform validatein CI, before anyone reviews HCL style by eye.
Common Mistakes
- Hardcoding what should be a variable. The first hardcoded region or instance size makes the module a renamed copy, not an abstraction.
- Variables for everything. The opposite failure: exposing every resource argument as an input means the module owns no decisions. Sensible defaults plus a small surface is the goal.
counton lists. Removing item 1 of 3 shifts indices 2 and 3, so Terraform plans to destroy and recreate them. Usefor_eachwith stable keys.- Missing
required_version. A module written against 1.5 syntax fails with cryptic parse errors on a 1.0 CLI instead of a clean “upgrade Terraform”. - No outputs. Consumers can’t reference what you didn’t export, so they hardcode IDs — and the module contract leaks.
latestprovider versions. An unpinned provider upgrades when it wants to, not when you want to.~> 5.0takes patches, skips majors.depends_onas decoration. If data already flows module → module, explicit ordering adds nothing but serialized applies.- Pinning a registry module with
?ref=.?ref=is git-source syntax; registry modules pin withversion = "x.y.z". Mixing them gives you a module that failsinitor ignores your pin.
Further Reading
Frequently Asked Questions
Should I use the Terraform Registry or private Git repos?
Registry for modules others can benefit from — it gives you source/version ergonomics, generated docs, and version constraints. Private Git repos for organization-specific modules until they mature; they pin with ?ref= tags or SHAs instead. A common path is git-source first, registry once the API stabilizes.
How do I migrate a resource from count to for_each?
Replace count = length(var.x) with for_each = toset(var.x) (keys become the values themselves), swap count.index references for each.value, then add one moved block per existing index so Terraform reuses the live resources instead of recreating them:
moved {
from = aws_subnet.public[0]
to = aws_subnet.public["10.0.1.0/24"]
}
Note you can't run count and for_each versions of the same resource block side by side — it's a declaration conflict. You change the block and migrate state with moved, not coexist.
Should modules manage their own state?
No. Modules define resources; the root configuration that calls them owns the state backend. Keeping state out of modules is what makes them portable across backends and environments.
Should I write my own VPC module or use terraform-aws-modules?
Start with terraform-aws-modules/vpc — it's battle-tested across thousands of environments and saves you from edge cases you haven't met yet. Write your own when its options become the thing you're fighting: unusual tagging policies, non-standard routing, or an internal contract your organization enforces. The example in this guide is deliberately minimal so the mechanics are visible; it's a learning scaffold, not a production replacement.
How do I share modules across teams without a registry?
Git sources with ?ref= tags are the low-friction path — one terraform-modules repo with a directory per module works well. Graduate to a private registry (HCP Terraform or self-hosted) when you need version constraints, generated docs, or discoverability across many teams.
How do I see what a module will create before using it?
Run terraform plan inside its examples/ directory — the example instantiates the module with defaults, so the plan is the full resource graph with no consumer needed. For inspecting values, terraform console evaluates expressions like module.vpc.private_subnet_ids interactively once init has run.
Related Resources
Infrastructure as Code — Terraform and Pulumi
A practical guide to infrastructure as code: declarative vs imperative, state management, modules, environments, testing, and a safe CI/CD workflow.
GuideTerraform Best Practices: Modules, State, and Workspaces
A practical guide to Terraform best practices: module design, remote state management, workspaces, and security for production-grade infrastructure as code.
GuideDeployment Strategies Compared: Blue-Green, Canary, Rolling
A practical guide to deployment strategies: rolling, blue-green, canary, shadow and feature flags. Minimize risk and rollback time when releasing to production.
RecipeBuild a Custom Terraform Provider with Python
Extend Terraform with a custom provider using Python and the terraform-plugin-framework to manage external resources.
DocInfrastructure as Code Review Template
A template for reviewing Terraform and CloudFormation infrastructure code.
GuideComplete Guide to AWS Cost Optimization
Reduce AWS spend 30-50% with EC2 right-sizing, Spot, Reserved Instances, Savings Plans, S3 lifecycle, RDS, networking and cost monitoring.