Now Available: env zero Environment Tagging

The Terraform registry is an essential asset for every Terraform user. In this blog, I’ll provide a practical guide for how it can be best used and explore ways you can leverage it to streamline and standardize cloud infrastructure provisioning.

What is Terraform Registry?

The Terraform registry is an official repository for Terraform modules and providers hosted by HashiCorp. 

The registry serves as a central hub for individuals and organizations looking to publish, discover, and use Terraform modules and providers to automate the setup and management of cloud infrastructure. 

Modules in the Terraform registry function similarly to libraries in a programming language, allowing reuse instead of requiring you to build your own from scratch. 

Providers are basically plugins for Terraform to communicate with cloud providers like AWS, Azure, Google Cloud, etc., to provision or manage your infrastructure.

The benefits of using the Terraform registry include:

1. Efficiency: You can utilize the registry to save time by using pre-existing modules instead of starting from scratch. 
   For example, you can import a pre-built Terraform EKS module from the registry, saving the time and effort of writing the EKS config.

2. Simplicity: Using the registry abstracts complex resource details simplifies your configuration.
   For instance, deploying a VPC manually requires configuring numerous components, including subnets, gateways, etc. However, utilizing a VPC module smooths the experience by abstracting the intricate configurations.

3. Peace of mind: The Terraform registry classifies modules and providers with badges (official, partner, community) to denote their source and level of verification. Moreover, the registry holds information about downloads, as an indication of usage and trustworthiness. 

Having this information available will help you identify popular, verified, and community-vetted assets and help improve your IaC quality, reliability, and security. 

Terraform Public vs. TFC Private Registry

The official Terraform registry is public, and should not be confused with the private self-hosted registry offered by TFC, env0, and other IaC management platforms, or OSS modules like Citizen, Terrareg, etc.

The use cases for using a private registry include:

• Customization: Organizations can create and manage modules that are tailored to their internal use, specific infrastructure requirements and policies, etc.
• Security: Security and compliance requirements can necessitate some organizations to self-host their registry or/and some of its modules.
• Centralized Governance: A private registry can be used for management and versioning of modules specific to the organization, to support internal guidelines and processes.
  

Example: Using Terraform AWS S3 Module

To show how the Terraform registry works (and how you should be working with it), here is a hands-on example of how we can provision an S3 bucket with Server Side Encryption (SSE) and versioning - all enabled through the use of an S3 module in the Terraform Registry.

Generally speaking, there are four steps you should take to use providers and modules from the registry to spin up an infrastructure:

1. Search and Discover

Since we want to import Terraform S3 code, our initial step would be to find an AWS S3 module with the necessary parameters (SSE and versioning). First, click on Browse Modules to find a range of different modules in the registry.

In our case, we filter the results to the AWS provider, since we want to provision an AWS S3 bucket.

2. Review and Select

Once we have found a module that meets our requirements, it is important to review its documentation, usage examples, and version information. 

In our case, we can see that the module is published by terraform-aws-modules, which is supported by the HashiCorp community. To further examine our parameters, we need to check the Inputs section.

Scroll down to check whether the module allows us to define the SSE and versioning input variables so that we can add these parameters to our S3 module block configuration. We see that it allows us to define these parameters.

3. Import Module into Terraform Code

Use the module in your Terraform configuration by referencing it in your .tf files. Specify the source of the module (its URL in the Terraform Registry). The basic syntax looks like this:

module "module_name" {
  source  = "namespace/module/provider"
  version = "version"
  # Additional required variables
}

In our case, we have defined the S3 bucket module in our main.tf file.

module "s3-bucket" {
  source  = "terraform-aws-modules/s3-bucket/aws"
  version = "4.1.0"
  bucket = "env0"
  versioning = {
    enabled = true
  }
  server_side_encryption_configuration = {
    rule = {
      apply_server_side_encryption_by_default = {
        sse_algorithm = "AES256"
      }
    }
  }
}

4. Run Terraform workflow

Run [.code]terraform init[.code] to initialize the Terraform project, which will download and install the specified S3 module and AWS provider it depends on. 

Run the [.code]terraform plan[.code] command to review the changes Terraform will make to your infrastructure based on the S3 module we included. 

Then, use [.code]terraform apply[.code] to spin up the S3 bucket, confirming the changes with a ‘yes’. 

After running [.code]terraform apply[.code], we have successfully provisioned our S3 bucket (env0).

Note: Understanding Verified Modules
----
‍By default, only verified modules are shown in search results.
HashiCorp reviews the modules, ensuring they meet their standards for quality and reliability.
‍
Three major benefits of verified modules:

- Reviewed by HashiCorp: Closely examined by HashiCorp to ensure they follow best practices in code quality, documentation, and module design. The verified badge appears next to modules that are published by a verified source, like HashiCorp.
- Actively Maintained: Contributors typically actively maintain verified modules, ensuring that they stay up to date with the latest Terraform versions and features.
- Trust and Reliability: Choosing a verified module means you're using a trusted component that is more likely to be reliable and secure for your infrastructure.

Example: Publishing Modules to the Terraform Public Registry

The next example I want to share will focus on adding your own module to the public registry.
To do that, imagine you've written Terraform code for an S3 bucket and you want to transform this code into a module, to enhance its reusability inside and outside your organization.

First and foremost, create a repository in your public Github account with an appropriate naming convention. The naming convention for repositories should be [.code]terraform-<provider-name>-<module-name>[.code]. 

Therefore, with respect to the naming convention, the given name of the module is kept as [.code]terraform-aws-s3-sse-versioning[.code].

After pushing the source code, the next important step is to tag our repository. To do that, head on to the Releases section and create a tag. We have created the tag 1.0.0. Select Publish Release after creating the tag.

To begin publishing your modules, navigate to the Terraform Registry and click on Publish Modules under the Publish dropdown. You will then be prompted to Sign In with GitHub. This step allows the Terraform registry to access and identify the repository where your module source code is located.

After authorizing your own registry with Github, (selecting Publish dropdown > Module), you’ll be prompted to select the repository for hosting the module. Choose the repo and select Publish Module.

Bingo! Within a few minutes, we can see that our module has been successfully published, which means you can pick up your existing terraform code and package it into a module.

env zero Private Registry

env zero offers two types of registries for your organization: Provider Registry and Module Registry.

Provider Registry

env zero Provider Registry is a feature that allows you to privately share and reuse Terraform providers within your organization. 

You can easily switch between different versions of the code with minimal changes. Moreover, within env0, authentication is seamlessly managed for you, requiring no extra steps or configurations.

It also enforces the use of RBAC policies to control access to the Provider Registry. For instance, users with “View Providers” have the privilege to only view the providers. As an administrator, you always have “Create and Edit” access.

Check out the Provider registry for more detailed information.

Module Registry

The env zero Module Registry is a private registry for Terraform modules, allowing you to privately share and reuse Terraform modules within your organization.

Creating and using a module with the env zero module registry is seamless.

1. In your env zero account, go to [.code]Registry > Modules section > Create a Module[.code].
2. Give a suitable name for your module and the provider name.

3. I’m leaving the tag prefix (env zero auto-detected repo tag, in our case) and module folder (as the module resides in the root folder) empty.

4. The private module has now been created, and the team in your organization can import the module using the appropriate format.

env zero has also rolled out its new CI Testing feature. This feature was specifically designed by env zero to incorporate the OpenTofu testing capabilities into their platform.

The OpenTofu testing feature ([.code]tofu test[.code]), validates your OpenTofu configuration (including modules) by creating real infrastructure and checking if the conditions (or assertions) for that provisioned infrastructure are met.

Tests suite is defined in *.tftest.hcl files and the run blocks should be where the test conditions are defined.

Each test run validates the following:

• All assertions pass
• None of the checks are failing
• The tofu plan/apply has finished successfully

For more information on how this feature works, check out the [.code]tofu test[.code] command.

Coming back to CI testing, this feature effortlessly integrates OpenTofu testing capabilities into your private module registry, acting as a health monitor for your modules and initiating tests with every change. Information is conveniently available in env zero and your VCS for a clear overview of your module status. This facilitates the mitigation of issues early on, improving user experience.

Frequently Asked Questions/FAQs

Q. Is the Terraform registry free?

Yes, Terraform's public registry is freely accessible to the general public, offering a wide range of providers and modules. 

Q. Is the Terraform registry open-sourced?‍

No. Terraform Registry is the property of HashiCorp. In 2023, following the license change of HashiCorp’s product, the company also updated the registry’s terms of service (TOS) for the Terraform Registry to limit its community usage, stating that: “You may download or copy the Content (and other items displayed on the Services for download) for personal non-commercial use only, provided that you maintain all copyright and other notices contained in such content.”‍

Q.  What registry is used by OpenTofu? 

OpenTofu, Linux-foundation’s backed open-source Terraform alternative,  maintains its own registry, offering access to a comprehensive set of providers and modules, including all the providers and modules currently available in the Terraform registry. 

Q. How do I publish a module to a private registry?

As discussed in this blog, publishing a module to a private registry is similar to publishing a module to the public registry. The process involves writing your own module, pushing it to your VCS (Version Control System) with semantic version tags, and then integrating your VCS with the private registry (e.g., Terraform Cloud, env0, etc.) to select and publish the module.

Q. Can I use multiple providers in Terraform, and where are providers stored?

Yes, you can use multiple providers in Terraform for provisioning infrastructure in different cloud environments. After you define the provider within the terraform block and run terraform init, Terraform downloads the respective provider plugin and stores it in the .terraform/providers folder in your local environment.

Q. What is the difference between Terraform providers and modules?

Providers in Terraform are plugins that interface with the API of a service provider (such as AWS, Google Cloud Platform, Azure, etc.) to manage resources. For example, the AWS provider can manage resources like virtual machines, databases, networks, etc., on AWS.

Modules, on the other hand,  can be used to create reusable templates for common infrastructure setups, like setting up a VPC network architecture on AWS.

In the dynamic world of DevOps, two powerhouse tools often dominate the discussion: Ansible and Terraform. Each brings unique strengths to the table, with Ansible excelling in configuration management and Terraform in robust infrastructure provisioning.

In this blog post, we will compare the two tools, and then consider examples of how to use them together. This demo will help illustrate how integrating Ansible and Terraform can lead to a more efficient and comprehensive approach to infrastructure management.

Video Walk-through

‍Requirements

• A free GitLab account
• Access to an AWS account, we’ll be running within the 12-month free tier
• A free envzero account

TL;DR: You can find the main repo here.

What is Ansible?

Ansible, a robust automation tool, simplifies complex IT tasks in cloud infrastructure and beyond, interfacing seamlessly with cloud provider APIs. It stands out as a key player in the world of infrastructure automation, particularly for its ability to manage software and deploy infrastructure resources, including network devices and virtual machines. 

Unlike traditional automation frameworks, Ansible automates these tasks without requiring agent software, making it a favorite among DevOps teams.

It is a command-line tool designed to manage infrastructure and deploy infrastructure resources, including network devices and load balancers.

‍

Why Consider Ansible as an Alternative to Terraform?

While Terraform is known for its provisioning infrastructure capabilities, employing the HashiCorp Configuration Language (HCL) for defining infrastructure as code (IaC), Ansible brings its strength as a configuration management tool. Ansible automates provisioning and ongoing maintenance of existing infrastructure, adapting to new cloud infrastructure needs. This makes it a versatile choice for IaC management, especially for Day 2 operations involving configuration changes and application deployment.

Terraform excels with its declarative approach within Terraform configuration, utilizing an immutable infrastructure approach which is highly effective for infrastructure cloud provisioning. 

However, when it comes to the practical aspects of configuration management, such as applying updates to an existing database, an imperative approach is often more suitable. Ansible exemplifies the imperative philosophy by ordering configuration steps sequentially, outlining exactly what needs to be done at each stage.

Other Tools in the Mix

In the realm of orchestration and configuration management, other tools like Chef, Puppet, and SaltStack also play significant roles. These configuration management tools each have their unique features, but Ansible stands out for its simplicity and efficiency, especially when it comes to infrastructure lifecycle management.

Ansible Use Cases

1. Cloud Infrastructure Visibility

Ansible serves as a powerful tool for gaining insights into the same infrastructure deployed across multiple environments. Especially useful in environments where multiple IT admins work independently, Ansible can track and report on your cloud usage. This approach is particularly beneficial for established, or "brownfield," environments. It's a low-risk, high-value use case since it's read-only and doesn't require altering your production environment.

2. Compliance Management

Ansible isn't just about infrastructure management; it's also about ensuring compliance across your cloud environments, such as the major cloud providers: Google Cloud Platform (GCP), AWS, and Azure. It can enforce policies like IAM rules and standardize experiences across different public clouds. Ansible is adept at managing both mutable and immutable infrastructures, ensuring that instances adhere to tagging policies for streamlined billing and auditing, and even shutting down non-compliant resources.

3. Business Continuity

Keeping your digital operations running smoothly is crucial, and Ansible plays a key role in this. It aids in transferring and duplicating resources off-cloud, automating backup policies, and managing disruptions. By building automation strategies with Ansible, you can ensure that your business remains resilient in the face of failures or other disruptions.

4. Cloud Operations and Lifecycle Management

Ansible excels in automating day-to-day cloud operations, which includes deploying applications, managing CI/CD pipelines, and handling OS patching and maintenance, reflecting on the underlying infrastructure. This automation extends to lifecycle management, ensuring that your cloud resources are always up-to-date and functioning optimally.

By automating these routine tasks, Ansible frees up your team to focus on more strategic initiatives.

Ansible vs. Terraform

It's not always a straight-up Terraform vs. Ansible showdown, but stacking both tools against one another sheds some light on the differences between the two of them.

Infrastructure Provisioning vs Configuration Management

While Terraform is a powerhouse in provisioning infrastructure, and creating new cloud infrastructure from scratch, Ansible shines in configuration management. Ansible's automation platform allows for detailed management of both the setup and ongoing maintenance of infrastructure, making it ideal for managing infrastructure changes over time.

Provisioning Resources

Terraform's strength lies in building and changing infrastructure efficiently. It's designed to create an immutable infrastructure where changes are made by rebuilding the infrastructure from a baseline. Ansible complements this by managing the configuration of these resources, ensuring they remain in the desired state.

Community/Ecosystem and Integrations

Both Terraform and Ansible boast vibrant communities and ecosystems. But when you stack Terraform vs. Ansible head to head, Terraform is often preferred for infrastructure provisioning in cloud environments, while Ansible is celebrated for its configuration management capabilities and as a cross-domain automation solution.

Chart Comparison

Using Ansible and Terraform Together

Example 1: VM and Infrastructure

The concept of immutable infrastructure, championed by Terraform, is an ideal state many strive for. However, the reality often differs. Many organizations have adopted Infrastructure-as-Code practices but still need to maintain and configure servers in the traditional way.

Terraform excels in defining infrastructure, but the virtual machines and resources it creates often require ongoing configuration and maintenance.

This is where Ansible comes into play.

Known for its simplicity in configuration management, Ansible complements Terraform by handling the post-deployment configuration and maintenance of resources. Integrating Ansible with Terraform, especially through platforms like env0, allows for smooth IaC rollout. This includes deployment, configuration, and maintenance, all managed as code within the same source control.

In a typical setup, Terraform is used to create infrastructure like an EC2 instance on AWS. Following the infrastructure setup, Ansible takes over with a Custom Flow to configure the deployed instance, employing an Ansible playbook. This process is streamlined and efficient, allowing for repeated runs without duplicating resources.

Instead, Ansible updates the existing host as needed.

The integration is facilitated by an env0.yml file, which orchestrates the entire process. This file ensures that before running [.code]terraform plan[.code], necessary preparations like SSH key retrieval are made, considering that Ansible will connect to the new machine via SSH.

The installation of Ansible and the creation of an inventory file for the EC2 instance are also part of this flow. The process concludes with running the Ansible playbook, which configures the host, leveraging environment variables for seamless execution. To dig deeper check out this blog post Terraform + Ansible = Total Flexibility.

This approach, while simple in this example, lays the foundation for more complex scenarios. The key lies in the env0.yml file, which orchestrates the custom flow, neatly integrating Terraform's infrastructure provisioning capabilities with Ansible's ability at configuration management tasks. This combination simplifies managing infrastructure resources.

It also ensures that they are consistently configured and maintained, aligning with the best practices of Infrastructure-as-Code. This methodology can be extended and adapted to more complex setups, demonstrating the elasticity of combining Ansible and Terraform in modern cloud environments.

Example 2: Day n Ops

The Asian Development Bank's approach – using Terraform for Day 0 operations and Ansible for Day 1 to Day n operations – illustrates an interesting cloud infrastructure management strategy. By leveraging Terraform at the outset, they efficiently provision cloud infrastructure.

Once this foundation is established, the focus shifts to Ansible, which they utilize for ongoing configuration management and maintenance from Day 1 onwards.

This combination streamlines the entire lifecycle of their infrastructure while securing consistency, reliability, and agility in their operations. It's a testament to the power of integrating these two tools, where Terraform's strength in initial setup complements Ansible's prowess in subsequent management.

This approach offers a glimpse into best practices for cloud infrastructure management, and more insights can be gleaned from their detailed video explanation. You can learn more in this video.

Demo Time!

Let's now examine our demo example where we combine Terraform and Ansible together.

We'll start by constructing an AWS EC2 instance using Terraform and then deploy a Docker engine on it with Ansible, followed by launching a Jenkins container. Utilizing env0's custom flows, we'll orchestrate Terraform and Ansible to create the setup.

In this scenario, our objective is to swiftly set up a Jenkins server for testing and utilize env0's capability to automatically decommission the server upon reaching a predetermined time-to-live (TTL). This TTL functionality is a valuable asset for cost-saving, ensuring that the server doesn't incur unnecessary expenses if left running inadvertently.

Watch the video at the top of this blog post to see how we create a project, a template, and an environment in envzero that spins up our Jenkins machine. I've included the configuration of Terraform, Ansible, and the envzero custom flow in the following sections.

Custom Flow with env0

envzero provides the capability to implement various hooks at different stages of a Terraform execution, which proves to be highly beneficial for integrating third-party tools like Checkov for security checks or, as in our scenario, with Ansible. 

To facilitate this, you simply need to create a file named env0.yml and position it at the root of your repository. For details on the specific hooks available for Terraform, refer to the envzero documentation.

env0.yml file

deploy:
  steps:
    terraformOutput:
      after:
        - terraform output -raw private_key > /tmp/myKey.pem
        - chmod 400 /tmp/myKey.pem
        - sed -i "s/[placeholder_app]/$(terraform output -raw public_ip)/g" Ansible/inventory
        - pip3 install --user ansible
        - ls -lah
        - cat Ansible/inventory
        - cd Ansible && ansible-playbook --private-key /tmp/myKey.pem -i inventory jenkinsPlaybook.yaml

The env0.yml file defines a set of custom workflow steps that are executed during the deployment process managed by env0. This particular configuration specifies actions to be performed after the Terraform output has been generated. Here's what each step in the deploy.steps.terraformOutput.after section does:

Private Key Extraction:

[.code]terraform output -raw private_key > /tmp/myKey.pem[.code]: This command extracts the private key from Terraform's output and saves it to a temporary file on the deployment environment, allowing for secure SSH access to the provisioned resources.

- File Permission Adjustment:

[.code]chmod 400 /tmp/myKey.pem[.code] This command changes the file permissions of the private key to ensure that it is read-only by the owner, which is a security best practice for SSH keys.

- Dynamic Inventory Update for Ansible:

[.code]sed -i "s/[placeholder_app]/$(terraform output -raw public_ip)/g" Ansible/inventory[.code]: This command replaces a placeholder in the Ansible inventory file with the public IP address of the provisioned infrastructure, outputted by Terraform. This dynamically updates the inventory to target the newly created EC2 instance.

- Ansible Installation:

[.code]pip3 install --user ansible[.code]: This command installs Ansible using pip, Python's package manager, ensuring that the necessary tool for configuration management is present in the environment.

- Directory and File Verification:

[.code]ls -lah[.code]: Lists all files and their permissions in the current directory to verify the presence and permissions of required files, such as the private key and Ansible inventory.

[.code]cat Ansible/inventory[.code]: Displays the contents of the Ansible inventory file (hosts file), which is useful for debugging purposes to confirm that the inventory has been updated correctly.

- Ansible Playbook Execution:

[.code]cd Ansible && ansible-playbook --private-key /tmp/myKey.pem -i inventory jenkinsPlaybook.yaml[.code]: This command changes the directory to the Ansible folder and runs the Ansible playbook, which is tasked with setting up Jenkins on the EC2 instance. It uses the private key saved earlier for SSH access and references the updated inventory file.

In essence, the env0.yml file orchestrates the deployment process by integrating Terraform and Ansible. It ensures that after the infrastructure is provisioned with Terraform, Ansible is correctly set up and then executed to configure the infrastructure (in this case, to install and run a Jenkins container). 

Terraform Configuration

Let's now dive into the Terraform configuration.

main.tf file

Let’s break down the main.tf file to understand it. You can find it in the GitHub repo.

This Terraform code is designed to both provision and set up a network infrastructure on AWS, including an EC2 instance that could be used for a Jenkins server. Let's break it down.

- Provider Configuration:

The required_providers block specifies that this Terraform code uses the AWS provider and the TLS provider from HashiCorp, with specific versions defined for both.

The provider block configures the AWS provider with a region that is specified by a variable, allowing you to define the region dynamically when you run Terraform.

- VPC Creation:

The resource "aws_vpc" "env0" block creates a new Virtual Private Cloud (VPC) with DNS hostnames enabled. It uses variables to set the CIDR block and tags, allowing for customization of the VPC's address space and naming.

- Subnet Creation:

The resource "aws_subnet" "env0" block creates a subnet within the VPC created earlier, with a CIDR block defined by a variable.

- Security Group Setup:

The resource "aws_security_group" "env0" block sets up a security group for the VPC, defining ingress rules to allow incoming traffic on specific ports (22 for SSH, 8080 for web access, and 50000 which is typically used by Jenkins for agent connections) and a general egress rule to allow all outgoing traffic.

- Internet Gateway:

The resource "aws_internet_gateway" "env0" block attaches an internet gateway to the VPC, which is necessary for the VPC to communicate with the internet.

- Routing Table:

The resource "aws_route_table" "env0" and resource "aws_route_table_association" "env0" blocks create a route table for the VPC and associate it with the subnet. This includes a route to direct all outbound traffic to the internet gateway.

- AMI Data Source:

The data "aws_ami" "ubuntu" block looks up the latest Ubuntu 20.04 AMI that is owned by Canonical, ensuring the instance uses a recent and supported OS image.

- Elastic IP and Association:

The resource "aws_eip" "env0" and resource "aws_eip_association" "env0" blocks provision an Elastic IP (EIP) and associate it with the EC2 instance, giving it a static, public IP address.

- EC2 Instance Provisioning:

The resource "aws_instance" "env0" block provisions an EC2 instance with the found Ubuntu AMI, the instance type specified by a variable, and associates the instance with the previously created security group and subnet.

- TLS Key Pair Creation:

The resource "tls_private_key" "env0" block creates an RSA private key used for secure communication.AWS Key Pair:The resource "aws_key_pair" "env0" block uploads the public key from the TLS private key to AWS to allow secure SSH access to the EC2 instance.

In summary, this Terraform code sets up all the necessary components for a secure, accessible, and isolated environment on AWS for a Jenkins instance.

The use of variables and dynamic data sources like the AMI lookup makes the code reusable and adaptable. The EC2 instance is configured with an Elastic IP and the necessary security group rules, ready to have Docker and Jenkins installed and configured by Ansible. The setup leverages env0's custom flows for smoothly automated infrastructure management.

variables.tf file

Once again you can find the content of the variables.tf file in the repo.

Below we explain it in detail.The variables.tf file defines variables in a single file to use throughout all Terraform configuration files, allowing for more flexibility and code reusability. Here's a breakdown of each variable defined in the variables.tf file you provided.

- Prefix:
The [.code]prefix[.code] variable is intended to be a string that will be prepended to the names of most resources created by Terraform. It helps in identifying and organizing resources, especially when managing multiple environments.

- Region:
Specifies the AWS region where the resources will be created. It has a default value of [.code]us-east-1[.code]. This allows the user to set the region for resource deployment, and if not specified, it defaults to the US East (N. Virginia) region.

- Address Space:

The [.code]address_space[.code] defines the CIDR block for the Virtual Private Cloud (VPC). The default value is [.code]10.0.0.0/16[.code]. This address space can encompass multiple subnets. Changing this value after deployment will force Terraform to create a new VPC resource.

- Subnet Prefix:

Sets the CIDR block for a subnet within the VPC. The default is [.code]10.0.10.0/24[.code]. This defines the range of IP addresses that can be used within this subnet.

- Instance Type:

The [.code]instance_type[.code] determines the type of EC2 instance to be launched. The default is set to [.code]t2.micro[.code], which is a small, low-cost instance type ideal for testing and small-scale applications.

- AWS SSH Key:

The [.code]my_aws_key[.code] string variable holds the name of the AWS key pair that will be used for SSH access to the EC2 instances. The default value is [.code]mykey.pem[.code]. This key should be present in your AWS account to ensure successful SSH connections.

By using these variables, the Terraform configuration gets more dynamic and customizable. Users can easily change these parameters to suit their specific deployment needs without altering the main config files, making the code easier to maintain and scale.

outputs.tf file

output "url" {
  value = "http://${aws_eip.env0.public_dns}"
}
output "public_ip" {
  value = aws_eip.env0.public_ip
}
output "private_key" {
  value     = tls_private_key.env0.private_key_pem
  sensitive = true
}

The outputs.tf file defines output values that you can easily retrieve after your infrastructure is provisioned. These outputs can be helpful for understanding important attributes of the resources that Terraform manages, or for feeding these values into other tools and scripts.

Here’s what each output in the provided outputs.tf file represents:

- URL Output:

This output generates a URL for accessing the provisioned resource, specifically using the public DNS of the Elastic IP (EIP) associated with your AWS resource which is our EC2 instance running Jenkins. The URL is formed by concatenating "http://" with the public DNS of the aws_eip.envzero resource. This URL can be used to access the EC2 instance from a web browser or a tool that can interact with HTTP endpoints.

- Public IP Output:‍

Provides the public IP address of the aws_eip.env0 resource. The public IP address is essential for connecting to the EC2 instance over the internet, for instance, via SSH or other network protocols.

- Private Key Output:

Outputs the private key generated by the tls_private_key.env0 resource. This key is used for secure SSH access to the EC2 instance. The [.code]sensitive = true[.code] attribute means that Terraform will treat this output as sensitive information. When you run Terraform in the CLI, it will not display this sensitive output in the CLI output. This is crucial for security reasons, as private keys should be kept confidential and not exposed publicly.

Ansible Configuration

Now it's time to explore the Ansible configuration.

jenkinsPlaybook.yaml file

I’ll now explain the content of the jenkinsPlaybook.yaml found in our repo.

- Setting Up the Environment:

1. Install pip3 and unzip: The first task is about installing python3-pip and unzip using the apt module. It also ensures that the cache is updated. We're using a loop here to try five times in case of failure, with a delay of 5 seconds between tries.
2. Add Docker GPG apt Key: Here, we're adding the Docker GPG key to ensure the packages we install are authenticated and secure.
3. Add Docker Repository: This adds the Docker repository to your system's repository list. We're specifying Ubuntu's focal release and setting the state to 'present' to make sure it's added.

- Installing Docker:

Update apt and install docker-ce. This updates your package list and installs the Docker engine (docker-ce).

- Setting Up Docker for Python:

Install Docker module to control Docker containers using Python.

- Pulling Jenkins Docker Image:

Pulls the custom Jenkins Docker image samgabrail/jenkins-tf-vault-ansible:latest from the Docker hub.

- Preparing Jenkins Data Directory:

Changes file ownership, group and permissions, which sets up a directory (/home/ubuntu/jenkins_data) for Jenkins data. It also adjusts the ownership and permissions so Jenkins can use it.

- Launching Jenkins in Docker:

Finally, we're creating and starting the Jenkins container. It's set to use the previously pulled Docker image and maps ports 8080 and 50000 for web interface and agent connections. The Jenkins data directory is also mounted as a volume inside the container.

This playbook ensures your environment is ready, installs Docker, sets up Python to work with Docker, pulls a custom Jenkins image, prepares a data directory, and starts Jenkins in a Docker container. This way, you have a Jenkins server up and running smoothly. It's like setting up a mini data center for Jenkins with just a few lines of code!

inventory file

Lastly, let's take a look at the content of the inventory file below:

[all:children]
jenkins

[all:vars]
ansible_user=ubuntu
ansible_python_interpreter=/usr/bin/python3

[jenkins]
jenkinsvm ansible_host=[placeholder_app]

This is an Ansible inventory file, also known as ‘the hosts file,’ which is used to define and group the hosts (servers) that Ansible will manage. The inventory file is a crucial part of Ansible's configuration as it specifies the targets of Ansible playbooks. By default, it will be found in the directory ‘etc/ansible/hosts’. Let's break down each part of the file:

- [all:children]

This section is defining a group of groups. Here, it declares [.code]jenkins[.code] as a subgroup under the main group [.code]all[.code]. This structure is useful for organizing your hosts and can be leveraged for scaling and managing complex environments.

- [all:vars]

This section defines variables that apply to all the hosts in the [.code]all[.code] group, including its subgroups. In this case, it sets ansible_user as 'ubuntu', which means Ansible will use the 'ubuntu' user account for SSH connections to all the hosts. The [.code]ansible_python_interpreter[.code] is set to [.code]/usr/bin/python3[.code]', specifying the path to the Python interpreter on the managed hosts. This is important for environments where Python 3 is not the default Python version.

- [jenkins]

This is a specific group named [.code]jenkins[.code]. Groups in Ansible inventory files allow you to categorize and manage hosts based on characteristics, roles, or any other classification that suits your needs. For instance, all Jenkins servers can be grouped here.

- jenkinsvm ansible_host=[placeholder_app]

Within the [.code]jenkins[.code] group, this line defines a host named 'jenkinsvm'. The [.code]ansible_host[.code] variable is used to specify the IP address or hostname of the [.code]jenkinsvm[.code] server.

Here, [.code]placeholder_app[.code] is a placeholder that the envzero custom flow will replace with the actual IP address our Jenkins server took from the Terraform output. This allows Ansible to know where to connect to execute the playbook tasks.

Conclusion

In wrapping up our discussion on Ansible, Terraform, and their roles in infrastructure management, it's important to reiterate some key differences. Ansible has evolved to function effectively in this space while not originally designed as an Infrastructure as Code (IaC) tool.

It excels as a configuration management tool and works very well for ongoing maintenance of existing infrastructure. Terraform, on the other hand, is purpose-built for IaC, focusing on the provisioning and management of infrastructure from the ground up.

This distinction is crucial for understanding how each tool fits into the broader picture of IT infrastructure management and how they can be used together for a comprehensive approach as seen in our demo example.

As a provisioning tool, OpenTofu is emerging as the Terraform alternative for Infrastructure-as-Code. Positioned as a drop-in replacement for Terraform, OpenTofu extends the ability to efficiently manage infrastructure.

Whether using OpenTofu or Terraform with Ansible or another complementary tool, you can use envzero to orchestrate the entire Infrastructure-as-Code setup. Sign up for a demo today.

Terraform functions are essential for creating effective infrastructure code. They help automate tasks like generating resource names, calculating values, and managing data structures. 

In this blog post, we will explore using Terraform CLI's built-in functions in different ways, such as in locals, the console, output, and variables.

Understanding these functions is important for any DevOps or Infrastructure engineer who wants to improve their Infrastructure as Code (IaC) skills.

‍Disclaimer
‍ All Terraform functions discussed here work similarly in OpenTofu, the open-source Terraform alternative. However, in order to keep it simple and closer to what devops engineers are familiar with, we will refer to them as Terraform functions.

What are Terraform Functions

Terraform functions are built-in features that help with simple management and manipulation of data within your Terraform configurations, enabling you to perform data transformations and ensure smooth infrastructure provisioning. 

Terraform's built-in functions include a variety of utilities to transform and combine values such as string formatting, arithmetic calculations, and working with lists and maps directly in your code.

Use Cases for Terraform Functions

Terraform functions are important for tasks such as variable interpolation, generating resource names, and applying conditional logic. Let us discuss a few of the use cases below.

• Concatenating Strings - You can generate unique resource names by appending environment names (e.g., "dev", "prod") to base names (e.g., "app-server"), resulting in names like "app-server-dev" and "app-server-prod".
• Splitting Strings - You can split a comma-separated variable like "key1,value1,key2,value2" into a list of individual items: ["key1", "value1", "key2", "value2"] using the split function name.
• Converting Data Types - You can convert a list of IP addresses to a set to remove duplicates and ensure each IP address is unique. Terraform's built-in functions support such data-type transformations.
• Merging Tags - You can combine tags from various resources into a single set using the [.code]merge[.code] function. This helps manage and apply consistent tagging across resources.
• Implementing Conditional Logic - You can set different instance types based on a variable, such as t2.micro for development and t2.large for production.
• Generating Timestamps - You can record the exact time of resource creation for auditing or tracking purposes.

Testing Functions with Terraform Console

Before you apply functions in your configuration, the Terraform console helps you test and try the functions in a CLI. It shows how functions behave with different inputs in real-time, allowing you to fix issues immediately. 

Here's how you can get started with the Terraform console:

Open your Bash or any command-line interface:

By using the Terraform console, you can quickly grasp the functionality of various Terraform functions and integrate them into your Terraform or OpenTofu configuration.

Basic Structure and Usage 

Functions in Terraform are used within expressions to perform various operations. The basic structure involves calling the function by name and passing the required arguments.

For example, the

[.code]upper(“hello from env0”)[.code] - the [.code]upper[.code] function converts the string to uppercase:

You can use functions in your configuration in various ways. Let us take a look at some of them. 

Locals

While working with Terraform locals, you can make use of functions to keep your configuration DRY (Don't Repeat Yourself), which makes it easier to manage and update values in one place. 

For example:

Here, the [.code]upper[.code] function sets [.code]formatted_name[.code] to "ENV0".

Resource Configuration

Functions can also be used directly within your resource configurations to set values dynamically.

For example:

In the code above, the [.code]upper[.code] function is used directly within the resource configuration to set the [.code]Name[.code] tag.

Variables

You can use functions within Terraform variables to set values based on other inputs dynamically. This flexibility allows you to transform and combine values as needed.

For example:

Here, the [.code]upper[.code] function sets the variable [.code]instance_name[.code] default value to "ENV0”.

Outputs

You can also use functions in the output block to display the expression results.

For example:

Here, the [.code]upper[.code] function call sets the [.code]output[.code] value to "ENV0".

Terraform Function Categories

Functions in Terraform or OpenTofu are organized into several categories.

String 

This category focuses on string-related functions, making it easier to construct and manipulate strings within your code. This can be particularly useful for naming resources, generating tags, and formatting output values.

For example: let us define [.code]var.instance_base_name[.code] for the base name of our instance and [.code]var.env[.code] for the environment name in variables in variables.tf. 

join(separator, list) 

([.code]join[.code]) function concatenates a list of strings into a single string using a specified separator.

To create an aws_instance resource name, we use the [.code]join[.code] function to combine [.code]instance_base_name[.code] and [.code]env[.code] variables with a hyphen separator, resulting in "webapp-production".

split(separator, string)

The ([.code]split[.code]) function splits a string into a list of substrings using a specified separator.

To split the name, we use the [.code]split[.code] function, which breaks [.code]local.joined_name[.code] (e.g., "webapp-production") into a list of substrings: ["webapp", "production"].

replace(string, substr, replacement)

The ([.code]replace[.code]) function replaces all occurrences of substr within string with replacement.

Here, the [.code]replace[.code] function changes the given string from"webapp-prodution" to "service-production" by replacing "webapp" with "service".

trimspace(string)

The ([.code]trimspace[.code]) function removes leading and trailing spaces from a string.

The [.code]trimspace[.code] function removes the leading and trailing spaces from the description, resulting in "This is a description with leading and trailing spaces".

Now, we will create a resource block using the locals from above,

Numeric 

Numeric-related functions help execute calculations on numeric values, such as rounding numbers or getting absolute values. These are helpful when adjusting resource configurations based on numeric input, such as sizing resources or calculating derived values.

For example: define [.code]var.desired_cpu[.code] for CPU allocation and [.code]var.desired_disk_size[.code] for disk size in variables.tf.

abs(number)

The ([.code]abs[.code]) function returns the absolute value of a given number.

Here, the ([.code]abs[.code]) function converts the desired_disk_size from -100 to its absolute value, 100.

ceil(number)

The ([.code]ceil[.code]) function rounds a number up to the nearest whole number.

Here, the ([.code]ceil[.code]) function rounds the [.code]desired_cpu[.code] from 3.7 up to 4.

floor(number)

The ([.code]floor[.code]) function rounds a number down to the nearest whole number.

Here, the ([.code]floor[.code]) function rounds down the  [.code]desired_cpu[.code] from 3.7 to 3.

These calculated values are used to define tags and configure an AWS instance's root block device. 

Now, we will create a resource block using the locals from above:

Collection 

This category focuses on handling and manipulating lists and maps, making working with complex data structures in your configurations easier. These functions are useful for counting elements, retrieving specific items, flattening nested lists, and merging maps.

For example: define [.code]var.security_groups[.code] to list all the security groups and [.code]var.additional_tags[.code] for adding additional tags to the resource in variables.tf.

length(list)

The ([.code]length[.code]) function returns the number of elements in a list.

Here, the ([.code]length[.code]) function counts the number of items within the [.code]security_groups[.code] variable, defining the total number of security groups.

element(list, index)

The ([.code]element[.code]) function retrieves a single element from a list by its index.

The ([.code]element[.code]) function retrieves the first item from the [.code]security_groups[.code] list, returning the first defined security group.

flatten(list)

The ([.code]flatten[.code]) function collapses a multi-dimensional list into a single-dimensional list.

The ([.code]flatten[.code]) function combines nested lists into a single list, resulting in ["env:production", "app:web", "tier:frontend", "region:us-east-2"]

merge(map1, map2, ...)

The ([.code]merge[.code]) function combines multiple maps into a single map.

In this example, the ([.code]merge[.code]) function combines the default tags with additional tags.

Now, we will create a resource block using the locals from above:

Date and Time 

This category focuses on date and time functions, allowing you to work with timestamps and schedule events. These functions are helpful for tasks like setting creation timestamps, scheduling backups, or calculating expiration dates.

timestamp()

The [.code]timestamp[.code] function name returns UTC's current date and time.

The [.code]timestamp[.code] function captures the current date and time when the configuration is applied and makes this timestamp available as both a local variable [.code]created_At[.code] and an output [.code]current_time[.code].

timeadd(timestamp, duration)

The [.code]timeadd[.code] function adds a duration to a timestamp, returning a new timestamp.

The [.code]timeadd[.code] function adds 168 hours (7 days) to the current timestamp to set the [.code]Backup_Schedule[.code] tag and the [.code]backup_time[.code] output.

Now, we will create a resource block using the locals from above:

Encoding 

This category focuses on encoding and decoding functions that help transform data between different formats, such as encoding strings to Base64 or decoding JSON strings into maps. These functions are useful for handling data in specific formats required by APIs or other services.

For example: define [.code]var.config_json[.code] for the configurations in JSON format in variables.tf.

‍

base64encode(string)

The [.code]base64encode[.code] function encodes a string to Base64 format.

The [.code]base64encode[.code] function encodes the [.code]original_string[.code] "This is a sample string." into Base64 format, resulting in [.code]encoded_string[.code].

jsondecode(string)

The [.code]jsondecode[.code] function decodes a JSON string into a map or list.

The [.code]jsondecode[.code] function decodes the JSON string stored in [.code]config_json[.code] into a map, resulting in [.code]decoded_config[.code].

Now, we will create a resource block using the locals from above:

For the complete code for all categories, please refer to this repository.

Working with Expressions

Expressions in Terraform help you handle and evaluate values in your configuration. Using conditional expressions, splat syntax, and functions to work with lists and maps can make your configurations more straightforward. 

These methods let you create resources based on conditions, manage collections, and retrieve specific values from lists and maps.

This approach simplifies the setup and ensures your infrastructure meets specific requirements.

Let's look at an example where we can use conditional expressions, splat syntax, and functions to manipulate data in our Terraform configurations.

Conditional Execution Using Ternary Operator

The ternary operator lets you choose between two values based on a condition.

If [.code]var.condition[.code] is true, the result is "SUCCESS" in uppercase; otherwise, it is "FAILURE" in lowercase. This helps dynamically set values based on conditions.

Accessing List Items With element

The [.code]element[.code] function retrieves an item from a list by index.

Here, the [.code]element[.code] function retrieves the second item (index 1) from [.code]var.instance_names[.code], which is "instance2". This helps you select specific items from a list.

Splat Syntax 

The splat syntax ([*]) allows you to access a specific attribute from all elements in a list of resources.

Here, the function retrieves the IDs of all instances created by the [.code]aws_instance.env0[.code] resource. This is useful for collecting all IDs and putting them into a list.

Joining Instance IDs into a Single String

The join function concatenates a list of strings into a single string with a specified separator.

Now, we will create a resource block using the locals from above:

In this example, the function combines all instance IDs into a single string, separated by commas. This is helpful for formatting lists into strings.

For the full code, refer to this GitHub repository.

‍Looping in Terraform

So far, we have learned the built-in functions in Terraform. However, there will be times when you’ll need to iterate over functions to solve slightly more complex problems by making use of looping mechanisms. 

Functions like [.code]count[.code], [.code]for_each[.code], and [.code]for[.code] let developers create and manage resources automatically.

for loop

The [.code]for[.code] loop in Terraform allows you to iterate over collections and transform their data. 

Let us take an example where the [.code]for[.code] loop transforms each tag key to uppercase and each tag value to lowercase, demonstrating how to iterate over a map and apply transformations.

for_each

The [.code]for_each[.code] construct allows you to create multiple instances of a resource based on the items in a map or set. This is useful for managing collections of resources with similar properties but unique values. 

In this example, [.code]for_each[.code] creates multiple AWS instances based on the server types defined in the [.code]servers[.code] variable.

count

The [.code]count[.code] meta-argument allows you to conditionally create resources based on a boolean expression. This is useful for managing resources that should only be created under specific conditions, such as deploying additional infrastructure for a staging environment.

For the full code, refer to this GitHub repository.

OpenTofu provider functions with env0

Until now, we've only discussed the shared functions of Terraform and OpenTofu. Now, let's look at OpenTofu provider functions, which add a unique extra capabilities by allowing providers to register and create custom functions.

When Terraform processes the [.code]required_providers[.code] block, OpenTofu asks each provider if they have any custom functions to add. These functions are then available in your module using the format :

And you can also use aliases for providers. 

Note that these functions are only available in the module where the provider is defined and are not shared with child modules.

Let's take an example using the following OpenTofu code to demonstrate how to use provider functions:

In this configuration, the [.code]corefunc[.code] provider is specified and pinned to version [.code]1.4.0[.code]. The provider is then initialized without any additional configuration. The [.code]str_camel[.code] function from the corefunc provider converts a string to kebab-case, removing any non-alphanumeric characters. 

You can use various functions from the corefunc provider, which you can find in corefunc functions documentation.

Next, let's use env zero to run this OpenTofu configuration. env0 is a powerful tool for automating and managing Terraform deployments, making running and managing your IaC easier.

Here's an overview of the steps to follow:

1. Create a new project in env zero and connect it to the repository containing the OpenTofu configuration.

2. env zero will automatically trigger the deployment, execute the OpenTofu code, and produce the desired output.

Using env zero to manage Terraform or OpenTofu deployments streamlines the process, allowing more focus on development and less on deployment.

Conclusion

We've covered how Terraform functions can simplify your infrastructure configurations. These functions enable you to create maintainable code by handling tasks like string manipulations, calculations, and data transformations.

Testing functions with the Terraform console ensure they work as expected before integrating them into your configurations.

Additionally, using loops and exploring OpenTofu provider functions with env zero workflow can further enhance your infrastructure management.

Frequently Asked Questions 

‍Q. What is the key function in Terraform?

In Terraform, a key function is any built-in function that performs a specific operation, such as generating timestamps, manipulating strings, or working with data types. Examples include [.code]timestamp()[.code], [.code]concat()[.code], and [.code]lookup[.code].

Q. What does [.code]${}[.code] mean in Terraform?

[.code]${}[.code] is used for interpolation in Terraform. It allows you to embed expressions within strings to reference variables, resource attributes, and call functions. For example: [.code]${var.instance_id}[.code] retrieves the value of [.code]instance_id[.code] from the [.code]var[.code] object.

Q. How do you check if a string contains a substring in Terraform?

To check if a string contains a substring, you can use the [.code]contains()[.code] function within a conditional expression:

Q. Can I create functions in Terraform?

No, you cannot create custom functions in Terraform. However, you can use existing built-in functions and modules to encapsulate reusable code.

Environment Tags are now available in env zero. You can attach key-value tags to any environment, like team=payments or cost-center=1042, to record ownership, cost attribution, or any other classification your project hierarchy doesn't express. Tags can be set in the UI or defined as code, filtered on across your estate, changed in bulk, and queried alongside your deployed infrastructure.

The request came from customers managing ownership outside the platform. One was maintaining 653 environment ownership overrides in a spreadsheet: correct on the day it was exported, stale by the end of the week, and impossible to filter or act on! Tags move that information into the environment itself.

What's new

Tag editing per environment

‍A Tags card in the environment's Settings tab, with suggestions drawn from values already used elsewhere in your organization. A key can hold one or more values, and an environment can carry up to 50 key-value pairs.

Tags as code

The tags attribute on env0_environment is supported in the Terraform provider as of v1.31.7:

resource "env0_environment" "payments_staging" {
  name        = "payments-staging"
  project_id  = env0_project.platform.id
  template_id = env0_template.service.id

  tags = {
    "team"        = "payments"
    "cost-center" = "1042"
  }
}

‍

Ownership is then reviewed in the same pull request as the rest of the environment definition, rather than tracked in a separate document.

Filtering and search

‍Filter the Explorer table by key=value, or search a project's environment list by tag. Explorer keeps the filter in the page URL, so a tag-filtered view is a link you can share.

Bulk changes

‍Add Tags and Remove Tags commands in the bulk operations wizard apply the same change across every selected environment, so retagging after a reorg is a single pass.

Introspect support

‍Tags feed into Introspect, so the labels you define are queryable against your actual deployed cloud resources. Note that in the asset graph, a key holding multiple values is flattened into one comma-separated string (payments,platform). This is lossless by design, and it is why the tag character set does not permit commas.

Permissions and limits

• Editing tags requires the Edit Environment Settings permission.
• Anyone who can view an environment can read its tags, so do not store secrets in them.
• Maximum of 50 key-value pairs per environment.

Availability

Environment Tags are available now in the UI and in the Terraform provider from v1.31.7. See the Environment Tags guide for full documentation.

We've already covered the mechanics of a single terraform import run, and separately looked at bulk import strategies like scripting and environment scans for larger batches. Both hold up fine once you already know which resources you're targeting. What neither one solves is the step before that: at the scale most brownfield environments actually present, the honest answer to “which resources need importing” is “we don't know, we inherited this.”

This piece picks up that specific problem: finding hundreds of unmanaged resources across accounts and clouds, deciding which ones matter first, and bringing them under management without turning the whole exercise into a six-month side project. That covers Terraform's own new bulk-discovery workflow, where it stops being enough on its own, and how CloudQuery and env zero's Cloud Compass close the gap between finding something and actually governing it.

Why single-resource import doesn't scale

terraform import and the declarative import block both assume you already know what you're importing: a resource address, a provider-specific ID, a resource block waiting to receive it. That's a reasonable assumption for one EC2 instance or one S3 bucket. It stops being reasonable once you're looking at an account that predates your IaC adoption by several years, or one that arrived through an acquisition with no documentation at all.

The actual bottleneck at that scale usually isn't the import command. It's discovery: knowing which resources exist, which of them are already managed somewhere, and which ones carry enough risk (an IAM role, a public S3 bucket, a security group with an open ingress rule) that they should jump the queue. Guessing your way through that with console tabs open across three accounts doesn't scale any better than running terraform import by hand for every resource does.

Terraform's native answer: list blocks and terraform query

Terraform 1.14, generally available since November 2025, shipped a native answer to part of this problem. A new .tfquery.hcl file type holds list blocks that query your provider for resources matching a filter, and the CLI gained a companion terraform query command to run those queries and, optionally, generate importable configuration from whatever it finds. Requirement-wise, resource-identity-based search needs Terraform 1.12 or newer, with the full workflow reaching general availability in 1.14.

A query file looks like this:

# find-unmanaged.tfquery.hcl
list "aws_instance" "unmanaged" {
  provider = aws
  limit    = 50

  config {
    region = "us-east-2"
    filter {
      name   = "tag:ManagedBy"
      values = ["unmanaged"]
    }
    filter {
      name   = "instance-state-name"
      values = ["running"]
    }
  }
}

Run terraform query and Terraform prints what it found, each result labeled with its list.. reference and resource identity. Add the -generate-config-out flag and it writes matching resource and import blocks, identity included, to a file you review and copy into your configuration before running terraform apply:

$ terraform query -generate-config-out=generated.tf

list.aws_instance.unmanaged: 12 results found
Generated configuration written to generated.tf

That's a real improvement over writing a dozen import blocks by hand, and it's worth knowing on its own merits. Two limits are worth flagging before you plan around it, though. First, list support has to exist for the specific resource type in that specific provider, so coverage varies and you should check your provider's documentation rather than assume it. Second, the polished side-by-side review experience (comparing what the query found against what's already tracked elsewhere) lives in the HCP Terraform UI and needs the cloud block configured to connect. Running terraform query itself doesn't require HCP Terraform, but the discovery it does is scoped to one provider configuration at a time.

Where native search runs out of room

For a single AWS account and a resource type your provider already supports, native query blocks are the right tool and probably the only one you need. The gap shows up at the scale that actually produces years of undocumented ClickOps: multiple clouds, SaaS platforms and identity providers alongside them, and a need to ask questions that don't map cleanly onto one provider's filter syntax, like “which storage resources across every cloud we run have no Terraform-managed tag at all.” A list block answers questions about one resource type in one provider. It doesn't give you a single place to ask that question once and get an answer back.

CloudQuery: one inventory across everything, not just what a single provider's list block supports

This is the problem CloudQuery, which merged with env zero in 2026, was built to solve before Terraform's own query workflow existed. CloudQuery syncs resource data from over 70 sources, cloud providers, GitHub, Okta, and more, into a single normalized, SQL-queryable layer. Every synced resource lands in a unified Asset Inventory with consistent schemas across providers, full-text search, and filters you can save and share. For anything a saved filter can't express (joins across resource types, aggregations, multi-condition logic) the SQL Console runs queries directly against the same data.

That matters for a question people actually ask, in one form or another, constantly: which resources across the whole environment have no tags at all, broken out by type? A resource nobody bothered to tag is usually a resource no pipeline, Terraform included, ever touched, which makes it a decent starting signal for “probably unmanaged.” Answering that by hand means maintaining a list of which of AWS's roughly 330 resource types actually support tags (the unified tagging API only covers about 140 of them), calling the rest through separate per-service APIs, and remembering that things like IAM role policies have to be excluded entirely since they're inline JSON attached to a role rather than a taggable resource in their own right. None of that per-service, per-account legwork exists once everything is already sitting in one normalized inventory. The sketch below isn't a documented CloudQuery query, table and column names depend on which sources you've synced and how, but it shows the shape of the question:

-- Illustrative shape only, not a documented CloudQuery query.
-- Substitute your actual synced table and column names.
select cloud, resource_type, count(*) as untagged_count
from 
where (tags is null or length(tags) = 0)
  and resource_type != 'aws_iam_role_policy'
group by cloud, resource_type
order by untagged_count desc;

For the real schema and supported query patterns, CloudQuery's SQL Console documentation is the source of record. The point holding up here isn't the exact syntax, it's that one query, run once, covers every connected cloud and account, where the manual version means hand-maintaining a taggable-type list and calling a different API per service, and a single list block still only covers one resource type in one provider.

From discovery to governed import with env zero

Finding a resource and safely bringing it under management are two different problems, and the second one is where env zero's own Cloud Compass comes in. Cloud Compass scans your connected cloud accounts, categorizes each resource by how it's actually been touched (ClickOps console changes, API or CLI calls, IaC operations) and scores each one on a severity scale that weighs both the mix of those operation types and how sensitive the service itself is. IAM, KMS, and EC2 carry more weight than a low-traffic S3 bucket, for the same reason a security team would care more about one than the other.

Select a resource in Cloud Compass and generate IaC code for it directly, Terraform or OpenTofu, ready to review and commit. Whether a given resource's config comes from Cloud Compass, from terraform query -generate-config-out, or from CloudQuery's inventory data feeding a hand-written import block, the resource lands in env zero the same way anything else does: through a normal PR-based plan. OPA policies evaluate the newly imported resource before it merges, the standard approval gates apply, and once it's in, drift detection owns it going forward the same as infrastructure env zero provisioned from scratch.

That's the practical version of the two companies' post-merger story: CloudQuery finds what exists across your entire environment, not just what one Terraform provider's list block happens to cover, and env zero is where that discovery turns into something governed rather than a one-time cleanup that drifts right back into ClickOps six months later.

A practical workflow for bulk import

1. Inventory everything first. Run a CloudQuery sync across your connected accounts and integrations before importing anything. You want the full picture, not just the corner of it one provider's query supports.
2. Prioritize by risk, not by convenience. Cloud Compass's severity scoring, or a manual pass weighting IAM, networking, and data stores above everything else, tells you which unmanaged resources represent real exposure versus which ones can wait.
3. Generate configuration, don't hand-write hundreds of resource blocks. Use Cloud Compass's generate-code action, terraform query -generate-config-out for resource types it supports, or both, depending on where a given resource showed up.
4. Land it through a normal PR. Import in batches small enough to review, not one sweeping commit. Each batch gets a plan, a policy check, and an approval, the same as any other change.
5. Re-run discovery and confirm. After each batch, check that the imported resources actually show up as IaC-managed rather than assuming the import succeeded because the apply didn't error.

Does OpenTofu support the same bulk query workflow?

Not yet, and this is one of the few places in this series where Terraform and OpenTofu genuinely diverge rather than matching each other feature for feature. an open feature request on OpenTofu's repository asks for a tofu query command and provider list resource support mirroring Terraform 1.14, and as of this writing it's still open. OpenTofu does support for_each on import blocks since version 1.7, which lets you bulk-import many resources of the same type through a single block once you already know their identifiers, the same for_each pattern that applies to resource and module blocks generally. What it doesn't yet have is the discovery half: a native way to ask a provider which resources exist that aren't in state yet. If you're on OpenTofu today, that gap is exactly where CloudQuery's provider-agnostic inventory carries more of the weight, since it doesn't depend on which engine eventually ships native query support.

Best practices for bulk import at scale

• Prioritize by service sensitivity first. An unmanaged IAM role or security group is worth importing before an unmanaged low-traffic storage bucket, regardless of which one your tooling happened to surface first.
• Import in reviewable batches. A single pull request with 200 new resource blocks defeats the purpose of routing imports through policy and approval in the first place.
• Pin provider versions across a batch. Resources imported under different provider versions can produce inconsistent generated configuration for the same resource type.
• Prefer import blocks over one-off CLI commands for anything beyond a handful of resources, so the batch is reviewable as a diff rather than a sequence of commands someone ran locally.
• Treat discovery as recurring, not a one-time project. Re-sync and re-scan on a schedule. New ClickOps resources accumulate the same way the original backlog did.

Frequently asked questions

Q. What's the difference between a single terraform import and bulk import?

A single import (terraform import or one import block) assumes you already know the resource address and provider ID you're targeting. Bulk import adds a discovery step in front of that, finding which resources exist and aren't yet managed, before any importing happens.

Q. Does terraform query require an HCP Terraform account?

No. terraform query runs from the Terraform CLI against your configured provider without needing HCP Terraform. HCP Terraform adds an optional UI for comparing query results against what's already tracked across your workspaces, but the underlying query and generate-config workflow works standalone.

Q. What is CloudQuery and how does it relate to env zero?

CloudQuery is a cloud asset inventory platform that syncs resource data from cloud providers and SaaS integrations into a single, SQL-queryable layer. env zero and CloudQuery merged in 2026, combining CloudQuery's discovery and inventory capabilities with env zero's IaC orchestration and governance.

Q. Should I use CloudQuery or Cloud Compass to find unmanaged resources?

They're complementary rather than competing. CloudQuery gives you a broad, cross-cloud, cross-integration inventory you can query with SQL. Cloud Compass builds on that with severity scoring specific to IaC coverage and a direct action to generate Terraform or OpenTofu code for a selected resource, inside the env zero workflow.

Q. Does OpenTofu support terraform query or list blocks?

Not currently. There's an open feature request on OpenTofu's GitHub repository asking for parity, but as of this writing OpenTofu's bulk-import story is for_each on import blocks (available since version 1.7), which simplifies importing many resources once you know their identifiers without adding a native discovery mechanism.

Q. How should I prioritize hundreds of unmanaged resources?

Weigh both the type of resource and how it's been touched. Services like IAM, KMS, and networking carry more risk than low-traffic storage or compute, and resources with a history of manual console changes carry more risk than ones only touched by scripts or existing IaC. Import the highest-risk combination first.

Key points

Single-resource import workflows, whether the CLI command or the declarative block, assume you already know what needs importing. At scale, that assumption is the actual bottleneck. Terraform's own terraform query workflow helps within a single provider's supported resource types. Beyond that, a provider-agnostic inventory like CloudQuery answers the broader question of what exists across your entire environment, and env zero's Cloud Compass turns that discovery into reviewed, policy-checked infrastructure rather than a spreadsheet that goes stale the day after you finish it.

What is Terraform 'for' Expression

Terraform "For Expression" is widely used, particularly in Terraform modules. A [.code]for[.code] expression allows you to create complex type values by transforming other complex type values.

This feature is not only beneficial in modules but also in your Main Infrastructure Code. Many infrastructure engineers are unaware of how to utilize these expressions and often end up with excessive repetitive code. In this guide, I will demonstrate an example to help you understand the advantages of using this expression, enabling you to write clean code and avoid repetitive declarations.

Terraform 'for' Expression Use Case with AWS

Imagine a scenario where we need to provision multiple AWS EC2 Instances but prefer not to place them within an Auto Scaling Group for management purposes. Additionally, we aim to incorporate a Load Balancer to evenly distribute inbound traffic among these multiple instances.

Getting Started

1. Let’s start by creating a local variable to specify how many nodes we want to deploy:

locals {
  nodes_count = 40
}

2. Let’s create the EC2 Instances based on the count. We will be using the Community Terraform Module to deploy the EC2 Instances.

module "lab_nodes" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.0.0"


  count = local.nodes_count


  name = "node-${count.index}"
}

3. Let’s create the Load Balancer and register our instances under our target group. We will be using the Community Terraform Module to deploy an Application Load Balancer. If you check the examples of usage of this module, you will see that each target has its own map specifying the Instance ID and the port. For example:

      targets = {
        my_ec2 = {
          target_id = aws_instance.this.id
          port      = 80
        },
        my_ec2_again = {
          target_id = aws_instance.this.id
          port      = 8080
        }
      }

Now, consider the scenario where we have already created 40 instances, and now we need to generate similar maps for each of these instances. Can you imagine the sheer size and repetition of the code in such a case? Not only would it be impractical, but it would also lack scalability. Now, let's picture a situation where the instance count needs to be increased from 40 to 100. Just think about the number of lines you would have to add in order to accommodate this change.

To avoid this to happen, we will be creating our ALB using the following code:

module "lab_alb" {
  source  = "terraform-aws-modules/alb/aws"
  version = "8.6.0"


  name               = "lab-alb"
  load_balancer_type = "application"


  target_groups = [
    {
      name             = "lab-tg"
      backend_protocol = "HTTP"
      backend_port     = 80
      target_type      = "instance"
      targets = {
        for i in range(local.nodes_count):
        i => {
          target_id = module.lab_nodes[i].id
          port      = 80
        }
      }
    }
  ]
}

Using 'for_each' instead of 'count'

1. In some cases it is better to use for_each instead of count, especially when you care about ordering. Also, [.code]for_each[.code] works based on maps or sets, so, we will be dynamically building a map from our nodes count.

locals {
  nodes_count = 40
  nodes_map = {
    for i in range(local.nodes_count):
      "node-${i}" => {}
  }
}

2. Let’s deploy the EC2 Servers:

module "lab_nodes" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.0.0"


  for_each = local.nodes_map


  name = each.key
}

3. Finally, let’s deploy our ALB:

module "lab_alb" {
  source  = "terraform-aws-modules/alb/aws"
  version = "8.6.0"


  name               = "lab-alb"
  load_balancer_type = "application"


  target_groups = [
    {
      name             = "lab-alb"
      backend_protocol = "HTTP"
      backend_port     = 80
      target_type      = "instance"
      targets = {
        for i, k in local.nodes_map :
        i => {
          target_id = module.lab_nodes[i].id
          port      = 80
        }
           }
    }
  ]
}

Here, we iterate over the local.nodes_map using a for loop. For each node, we dynamically assign the target_id as the corresponding EC2 instance ID, obtained from the module.lab_nodes output. Using dynamic code, we can effortlessly configure the ALB to target the appropriate instances based on the nodes_map.

Terraform 'for' Expression Code Explanation

In this Terraform code snippet, the for expressions are used to dynamically generate resources based on the specified logic. Let's break down what each section is doing:

1. [.code]locals[.code] block:‍

• [.code]nodes_count = 40[.code]: This defines a local variable [.code]nodes_count[.code] with a value of 40.
• [.code]nodes_map[.code]: This defines a local variable [.code]nodes_map[.code] as a map.
• The [.code]for[.code] expression [.code]for i in range(local.nodes_count)[.code] is used to iterate over a range of numbers from 0 to [.code]nodes_count - 1[.code].
• Inside the [.code]for[.code] expression, [.code]"node-${i}" => {}[.code] creates key-value pairs where the key is a string [.code]"node-${i}"[.code] and the value is an empty object [.code]{}[.code]. This effectively generates a map of 40 elements with keys like [.code]"node-0"[.code], [.code]"node-1"[.code], ..., [.code]"node-39"[.code].

2. ‍[.code] module "lab_nodes"[.code] block:

• This module creates EC2 instances using the Terraform AWS module [.code]terraform-aws-modules/ec2-instance[.code].
• The [.code]for_each = local.nodes_map[.code] meta argument tells Terraform to create an instance for each element in the [.code]nodes_map[.code] map.
• [.code]name = each.key[.code] sets the name of each instance to the key of the corresponding element in [.code]nodes_map[.code].

3. ‍[.code]module "lab_alb"[.code] block:

• This module creates an Application Load Balancer (ALB) using the Terraform AWS module [.code]terraform-aws-modules/alb[.code].
• [.code]name[.code] and [.code]load_balancer_type[.code] are basic configuration attributes.
• [.code]target_groups[.code] defines the target groups for the ALB.
• The [.code]for[.code] expression [.code] for i, k in local.nodes_map[.code] is used to iterate over each key-value pair in [.code]nodes_map[.code].
• Inside the [.code]for[.code] expression, [.code]i[.code] represents the key (e.g., [.code]"node-0"[.code], [.code]"node-1"[.code]) and [.code]k[.code] represents the corresponding value (an empty object in this case).
• The [.code]targets[.code] block within [.code]target_groups[.code] defines the targets for the target group.
• The [.code]for[.code] expression [.code]for i, k in local.nodes_map[.code] is used again to iterate over each key-value pair in [.code]nodes_map[.code].
• Inside the [.code]for[.code] expression, [.code]i[.code] represents the key (e.g., [.code]"node-0"[.code], [.code]"node-1"[.code]) and [.code]k[.code] represents the corresponding value (an empty object).
• [.code]i => { target_id = module.lab_nodes[i].id, port = 80 }[.code] specifies the target ID and port for each target based on the corresponding EC2 instance created by the [.code]module.lab_nodes[.code] module.

In summary, the for expressions in this code snippet are used to generate a dynamic number of EC2 instances and associate them with an ALB target group. The number of instances is determined by the [.code]nodes_count[.code] variable, and the instances are named according to the [.code]nodes_map[.code]. The ALB target group is configured to have targets corresponding to the EC2 instances created by the [.code]module.lab_nodes[.code] module.

Conclusion

In conclusion, the dynamic code showcased in this article demonstrates the power of Terraform's ability to handle dynamic infrastructure deployments. We can efficiently provision and configure resources by utilizing features like the for_each loop and dynamic mappings, making our deployments more flexible, scalable, and maintainable. Embracing dynamic code empowers us to build robust infrastructure as code solutions adapt to changing requirements and enable seamless collaboration among team members.

I strongly encourage you to learn and master the usage of these expressions in your code. Remember, there is always room for improvement. Investing time in crafting a well-written Terraform code will pay off in the future, as it will save you time and accelerate the progress of your infrastructure projects.

References:

Terraform For Expression Official Documentation:

 https://developer.hashicorp.com/terraform/language/expressions/for

Terraform Community Module for EC2:

https://registry.terraform.io/modules/terraform-aws-modules/ec2-instance/aws/latest

Terraform Community Module for ALB:

https://registry.terraform.io/modules/terraform-aws-modules/alb/aws/latest