Get Started With env zero

You run [.code]terraform apply[.code], it fails partway through, and the next command you run refuses to move: [.code]Error acquiring the state lock[.code]. Nothing is broken yet, but nothing will proceed either until the lock is cleared.

This guide covers what [.code]force-unlock[.code] actually does, when it is safe to use, how to find the lock ID on every major backend, and what to do if the command itself does not fix things.

What does terraform force-unlock do?

Terraform locks state before any operation that could write to it. The lock stops two processes from writing to the same state file at once, which is the most common way a state file gets corrupted. Locking happens automatically and silently on every plan and apply where the backend supports it. You will not see a message unless acquiring the lock takes longer than expected.

[.code]force-unlock[.code] is the manual override for when that automatic process gets stuck. It removes the lock record so a new operation can proceed. According to HashiCorp, the command does not modify your infrastructure, and on most backends it does not touch your state data either. It just clears the record that says the state is currently held.

Usage:

terraform force-unlock [options] LOCK_ID

The only option is [.code]-force[.code], which skips the yes or no confirmation prompt. That is useful inside a script or CI job where nothing is available to type “yes” into. Otherwise leave it off, since the confirmation step is the last chance to catch a mistake before you unlock something someone else is actively using.

The lock ID is not optional and is not guessable. Terraform prints it in the error message when a lock is already held, and [.code]force-unlock[.code] requires an exact match. Per HashiCorp's own documentation, the ID works as a nonce, a one-time verification token that ensures a lock and an unlock target the same lock. That is deliberate: you can only release a lock you can identify, not just any lock on the state file.

One thing worth flagging up front: on the local backend, a stuck lock can only be cleared by the same machine and user that created it. There is no separate process to force it from elsewhere, which is one more reason most teams move to a remote backend, such as the ones covered in this backend configuration guide, before this becomes a live problem.

When should you use the force-unlock command?

Treat [.code]force-unlock[.code] as a last resort, not a first response. If two operations are genuinely running against the same state at the same time, forcing a lock open defeats the entire purpose of state locking and can leave you with a corrupted state file. Only run it when you are certain the process that created the lock is no longer active.

A stuck lock usually traces back to one of a few causes:

• A [.code]terraform apply[.code] or [.code]plan[.code] was cancelled or errored mid-run, for example because a CI job timed out or someone hit Ctrl+C, so Terraform never reached the step where it releases the lock.
• The machine or build agent running Terraform lost its network connection to the backend before the lock could be released.
• The backend storage itself changed mid-operation, for example a Terraform run modifying firewall rules, private endpoints, or access policies on the very storage account that holds the state file.

If none of those match your situation and you are not sure why the lock exists, treat that as a reason to investigate before clearing it, not a reason to assume it is safe to force.

Where to find the lock ID for every backend

In most cases you will not need to go looking. The lock ID appears directly in the [.code]Error acquiring the state lock[.code] message, under the [.code]ID:[.code] field, alongside who holds it and when it was created. The backend-specific detail below matters mainly when you are troubleshooting secondhand, for example clearing a lock a teammate reported without a fresh error message in front of you.

Local backend

Terraform writes a [.code].terraform.tfstate.lock.info[.code] file next to the state file while an operation is in progress. It is a small JSON object containing the lock [.code]ID[.code], the operation type, and who created it. On clean exit, Terraform deletes this file automatically. As noted above, a lock created by one machine cannot be released by [.code]force-unlock[.code] from a different one.

Amazon S3

As of Terraform 1.11, the S3 backend supports native state locking through the [.code]use_lockfile[.code] argument, and no longer requires a separate DynamoDB table. Setting it to [.code]true[.code] tells Terraform to create a lock object in the same S3 bucket as your state, using conditional writes so only one process can create it at a time.

terraform {
  backend "s3" {
    bucket       = "mybucket"
    key          = "path/to/my/key"
    region       = "us-east-1"
    use_lockfile = true
  }
}

With native locking, the lock ID is whatever the error message reports; there is no separate table to query. If your configuration still uses the older [.code]dynamodb_table[.code] argument, note that HashiCorp has deprecated it in favor of [.code]use_lockfile[.code]. On that legacy path, the lock lives as an item in the DynamoDB table, keyed by a partition key named [.code]LockID[.code], and you can inspect it directly:

aws dynamodb get-item \
  --table-name your-lock-table \
  --key '{"LockID": {"S": "your-bucket/path/to/terraform.tfstate"}}'

Azure Blob Storage

Azure Blob Storage implements locking through native blob leases, with no extra backend configuration required. If a run is interrupted mid-apply, the lease can be left in place. The lock ID appears in the error message, but you can also inspect the lease state directly:

az storage blob show \
  --account-name YOUR_STORAGE_ACCOUNT \
  --container-name YOUR_CONTAINER \
  --name path/to/terraform.tfstate \
  --query 'properties.lease'

If [.code]force-unlock[.code] is not an option, for example the lock ID is unavailable, you can break the lease directly through the Azure CLI, which achieves the same result at the storage layer:

az storage blob lease break \
  --account-name YOUR_STORAGE_ACCOUNT \
  --container-name YOUR_CONTAINER \
  --blob-name path/to/terraform.tfstate

Google Cloud Storage

The GCS backend also locks natively with zero extra configuration. Terraform writes a lock object at [.code]/.tflock[.code] in the same bucket as your state, and the lock ID is the object's generation number, which is included in the error message. Deleting that object directly is the manual equivalent of [.code]force-unlock[.code] if the CLI command fails for some reason.

HCP Terraform and Terraform Enterprise

This is a common point of confusion: [.code]terraform force-unlock[.code] is a CLI command that works against backends where Terraform itself manages the lock file. HCP Terraform and Terraform Enterprise instead lock and unlock workspaces through their own UI and API, not the CLI command. In the workspace's Actions menu, you can select Lock workspace or Unlock workspace directly, or call the workspaces API endpoint to do the same thing from automation.

Consul

With the Consul backend, lock information lives in the Consul key-value store rather than in a file. You can list it with the [.code]consul kv get [.code] command, or query the same data through Consul's HTTP API.

Using terraform force-unlock: a worked example

1. Identify the lock ID from the error message. For example: [.code]Lock Info: ID: b8814894-4a5f-217b-e97b-c4f5c02a1f88[.code].
2. Confirm nobody else is running an operation against this state. Check your CI/CD dashboard, ask your team, or check the environment's deployment history if you are running on a platform that centralizes this, before assuming the lock is actually stale.
3. Run the command with the ID from step one: [.code]terraform force-unlock b8814894-4a5f-217b-e97b-c4f5c02a1f88[.code]. Confirm the prompt with [.code]yes[.code], or add [.code]-force[.code] if you are running this non-interactively.
4. Verify the fix by re-running the command that originally failed, such as [.code]terraform plan[.code]. If it proceeds past the locking step without error, the lock is cleared.

Unlocking remote state: alternatives to force-unlock

Wait instead of forcing: -lock-timeout

If two operations occasionally overlap for a few seconds, for example two CI jobs kicking off close together, [.code]force-unlock[.code] is the wrong tool. The [.code]-lock-timeout[.code] flag tells Terraform to wait for the lock to clear on its own instead of failing immediately:

terraform plan -lock-timeout=5m

This is worth setting as a default in CI pipelines that run plan, apply, or destroy operations back to back, so a brief overlap resolves itself instead of surfacing as a lock error at all.

Manual removal as a last resort

Occasionally [.code]force-unlock[.code] itself fails, usually because the backend is unreachable or credentials cannot reach the lock record. HashiCorp's guide to recovering state from backup covers this scenario directly. In that situation, the remaining options are backend-specific: delete the lock object from S3 or GCS, edit or remove the DynamoDB item, or break the Azure blob lease as shown above. All of these bypass Terraform entirely, so treat them with the same caution as [.code]force-unlock[.code] itself.

Coordinate before you unlock

Whichever method you use, confirm no other process is mid-write before you touch the lock, and consider pulling a backup first with [.code]terraform state pull[.code]. Never use [.code]-lock=false[.code] as a standing workaround for frequent lock errors. It disables the protection entirely rather than resolving whatever is causing the contention.

Troubleshooting force-unlock errors

The lock ID does not match

[.code]force-unlock[.code] will refuse an ID that does not match the current lock. This almost always means you are using a stale ID from an old error message. Re-run the failing command to get the current lock's ID and try again.

Permission errors during force-unlock

Clearing a lock requires write or delete access to wherever the lock record lives, for example [.code]s3:DeleteObject[.code] on the lock object, or the equivalent DynamoDB, GCS, or Azure permission. A permissions error here usually points to the credentials Terraform is running with, not the lock itself.

The same lock error comes back immediately

If you clear a lock and it reappears right away, something is still actively writing to that state. Stop and investigate before unlocking again. This pattern usually means step two of the worked example above was skipped.

Managing state locking at scale with env zero

Clearing a stuck lock by hand does not scale once a platform team is managing hundreds of environments across multiple backends. Ad Hoc Tasks in env zero let you run a command, including [.code]terraform force-unlock -force LOCK_ID[.code], directly on the environment's deployment container from the UI. That means resolving a stuck lock does not require local CLI access, a checked-out copy of the Terraform configuration, or direct credentials to the backend that holds the state.

By default, ad hoc tasks are restricted to organization administrators, since they allow arbitrary commands against a live deployment container. Teams that want to delegate lock-clearing to platform engineers without granting full admin access can do that with a custom role scoped to just that permission.

It is worth distinguishing this from Environment Locking in env zero, which is a separate, deliberate governance control rather than Terraform's automatic state lock. Locking an environment in env zero blocks deploys, destroys, plans, and drift detection outright, with a reason attached for anyone else who looks at it, and it stays in effect until someone with permission unlocks it. A Terraform state lock, by contrast, is transient by design and normally clears itself within seconds. If you are troubleshooting a “locked” environment in env zero and [.code]force-unlock[.code] does not seem relevant, this distinction is usually why.

Key takeaways

• [.code]terraform force-unlock LOCK_ID[.code] manually clears a stuck state lock. It does not touch your infrastructure, and on most backends it does not touch your state data either.
• Only use it when you are certain the process that created the lock is no longer running. Unlocking an active operation risks a corrupted state file.
• The lock ID is usually sitting right in the error message. You only need to hunt through backend-specific tooling when troubleshooting without that message in hand.
• S3 no longer needs DynamoDB for locking. [.code]use_lockfile = true[.code] has been the supported path since Terraform 1.11.
• [.code]-lock-timeout[.code] prevents most stuck-lock situations in CI before they happen, by waiting instead of failing immediately.

Frequently asked questions

Q. How do I fix a Terraform state lock?

Run [.code]terraform force-unlock LOCK_ID[.code], using the ID from the [.code]Error acquiring the state lock[.code] message. Only do this once you are certain no other operation is currently running against the same state.

Q. What is Terraform state locking for?

State locking prevents two operations from writing to the same state file at the same time, which is one of the most common causes of state corruption. Terraform acquires the lock automatically before any operation that could write state and releases it when the operation finishes.

Q. Can force-unlock corrupt my Terraform state?

[.code]force-unlock[.code] itself does not modify your infrastructure or your state data; it only removes the lock record. The risk is indirect: if you unlock a state that another process is actively writing to, that process and yours can both write at once, which can corrupt the state file.

Q. Does the S3 backend still need DynamoDB for state locking?

No. Since Terraform 1.11, the S3 backend supports native locking through [.code]use_lockfile = true[.code], using S3 conditional writes instead of a separate DynamoDB table. The older [.code]dynamodb_table[.code] argument still works but is deprecated.

Q. How do I avoid stuck state locks in the first place?

Avoid cancelling Terraform runs mid-operation, set [.code]-lock-timeout[.code] in CI so brief overlaps wait instead of failing, and use a platform that centralizes deployment history so you can quickly confirm whether a lock is stale before clearing it.

Hello, env zero fans! As some of you know, we have almost unlimited extensibility with 3rd party tools, using our custom workflows. You can hook in pretty much any tool, in any phase of the deployment. Today, we’re going to talk about how to prevent cloud misconfigurations before they start. We’re going to do this by chaining a tool in the deployment after the terraform plan phase. This is where our friends at Bridgecrew come in. Just like we at env zero have open-sourced the Terratag module of our platform, Bridgecrew has open-sourced Checkov!

Checkov

Checkov is a static code analysis tool for infrastructure-as-code. It scans cloud infrastructure managed in Terraform, Cloudformation, Kubernetes, Arm templates, or Serverless Framework and detects misconfigurations.

Setup

For illustration purposes, we’re going to use Bridgecrew’s demo application called TerraGoat. TerraGoat is Bridgecrew’s “Vulnerable by Design” Terraform repository. TerraGoat is a learning and training project that demonstrates how common configuration errors can find their way into production cloud environments.

DISCLAIMER: DO NOT ACTUALLY DEPLOY THIS APPLICATION INTO YOUR CLOUD INFRASTRUCTURE. IT IS PURPOSELY COMPROMISED.

I have created a template of TerraGoat inside of env zero and linked it to our Bridgecrew Demo project.

The only other thing we have to do is to actually call Checkov to do the check during the deployment. We need to do this after the Terraform plan phase, so that we have a plan to check. Here is what the env0.yml file will look like:

This adds 3 commands that run after the Terraform Plan, and before Terraform Apply. We put it here so that the Apply doesn’t run in case of failures. We don’t want to see the errors after the resources are applied. We want the deployment to fail if there are errors.

This command installs Checkov into our runtime environment using the pip3 package installer so we can run it against our Terraform plan.

This command essentially formats our .tf-plan file into tf.json so that it can be parsed and run against Checkov.

This command has a lot going on and is in 2 parts. First, it quietly executes Checkov against our tf.json (the reformatted tf.plan file) and looks for a 0 exit code. The double pipe || tells bash to only execute the 2nd command if the exit code of the first command is not 0. So if your Checkov results are clear, your deployment gets the 0 exit code and continues on with the deployment.

If not, then the second part of the command runs. Knowing if this part runs, it is because of a failure, we’re just going to format our error message here. We run Checkov again so we can pipe the error with the echoed error notification text to the console. The 1>&2 routs stdout to stderror, and the exit 1 code tells env zero that the stage failed, and to end the deployment run.

The env zero platform will parse the error, and give you the clear error printed on the Environment deployment page. But, if you want the full logs from Checkov, you can find those in the After: Terraform Plan deployment logs.

And that’s it! A little bit of YAML, and you’ve implemented Checkov to protect yourself against the deployment of misconfigured cloud resources. That is instantly added value to your organization by shifting the security left in your deployment process with env0.

You can find more information on Checkov here. You can find the open-source repository on GitHub. And be sure to see how you can automate your infrastructure security from commit to cloud at Bridgecrew.io.

‍

In a recent blog post, I discussed expanding the idea of “Feature branches” to “Feature environments”. Using Infrastructure-as-Code, we can create an environment for every feature we are working on, thereby giving us a more flexible, isolated development environment, and allowing us to test our code early in the development process.

In this post I’d like to continue down that path, and see how we can automatically create an environment for every pull request, and gain a number of advantages over traditional static staging or qa environments.

Pull Requests & Moving Beyond Static Staging

Pull requests are a well known and common workflow step for many development teams. We usually think of them as a way you “tell others about changes you've pushed”, and where you “can discuss and review the potential changes with collaborators and add follow-up commits before your changes are merged”.

PR’s are more than just a code review - they’re a milestone in a feature’s lifecycle and a way of saying “I’m ready for this to be shared”. Beyond sharing it for feedback with the wider team, this milestone is a critical time to ensure it functions exactly as expected as part of the whole application, including any potential infrastructure or configuration changes. However - just as we wouldn’t want our data migration to run on the shared database at this point, we also want to use dedicated test infrastructure.

Static VS Dynamic PR Environments

At this point, you might ask yourself - I’m already testing my code and infrastructure changes in our dev/qa/staging environment, why complicate things?

Well, there are a number of advantages to moving from traditional, static environments, to dynamic, per-pull-request environments:

• They’re Isolated and Dedicated - having a dedicated environment for each PR means no more confusion of which version or branch is currently in staging, and no coordinating between people who want to test different versions.
• Easier To Share - Because each PR has its own fully functional environment, non technical stakeholders can provide feedback on new features very early in the process. Developers can then iterate over this feedback - without interfering with work being done by other team members.
• No Wasted Resources - Because you’re only provisioning an environment when you actually need it for testing, you’re not wasting (or paying for) resources when you don’t actually need them.
• Removes Bottlenecks In The Release Cycle - Shared development and staging environments are notorious bottlenecks for development teams, especially when they are the first place where new code meets infrastructure. It’s not uncommon to see a queue of who is using the environment for testing their features. Your developers time shouldn’t be spent on waiting.

How Do You Actually Do It?

Ok, so “per pull request environments” is an awesome idea. How are we going to actually get there? There are a number of tools out there that can help you accomplish this task but in this post, I’ll be using env0, a first of it’s kind environment-as-a-service platform - not just to deploy the environments, but to manage them as well.

Your default assumption might be to just use your CI/CD platform to set up your environment. This works, but most CI/CD platforms are built for running short lived tasks, whereas an environment’s lifecycle extends beyond deploying it once: It needs to be updated, monitored, and in the end destroyed. Besides easily automating resource provisioning using Terraform, env zero will help me keep track of which environments are up, which ones have had issues, and will provide me a top level view of how my whole team is using these environments.

Besides env0, I’ll be using Github to host my code and open pull requests, and Github Actions to trigger my environment creation. I’ll be using the same code example from the previous post, which will be deployed on AWS.

If you’d like to try this out yourself, the prerequisites for this tutorial will be

• An env zero account (it’s free, just login)
• A Github account 
• An AWS account

Getting Our Hands Dirty

Step one to fully automating anything is to make sure we can run it manually, so you’ll want to get your system set up. In my case, I’ve followed env0’s getting started guide, and taken the key steps of creating my own organization, connecting my AWS account, and creating a template for my Terraform code.

Custom workflows to the rescue

In the example code I’ve used, I also ran a bash script before deploying the environment. We can easily make sure this code runs before our Terraform is applied, using env0’s Custom Flows feature. I’ve already done this in advance and I’ve put my code in the env0.yml file, in our Github repo.

Let ‘er Rip!

We are ready to launch our first environment! Remember - this is just a manual test, to see things are ready for automation.

In the case of env0, just go to your Project Templates pages, and click “Run now” on the template we’ve created before. In the next screen, you can validate your settings, and when you’re ready - click “Run”.

‍

‍

Great!

Integrating into CI/CD

Now that we know our environment management system will properly configure our environments, we need to make it run each time we open a pull request. For that, we’ll be using Github Actions to trigger env0’s CLI.

In order to create an environment on env zero from Github Actions, we need to create an API key for env0.

Next, we’ll need to save the API key and secret as Github Secrets, in the same manner we saved our AWS credentials to env0.

The final step of connecting everything, is telling Github how to trigger our environment deploy. We’ll add the following code to our codebase, in the file `.github/workflows/pr-environments.yml`.

name: "PR Environments"
on:
 pull_request:
   types: [opened, closed, reopened, synchronize]
jobs:
 env0_pr_environment:
   name: "PR Environment"
   runs-on: ubuntu-16.04
   env:
     ACTION: deploy
   steps:
     - name: Set Action
       if: github.event.action == 'closed'
       run: echo "::set-env name=ACTION::destroy"
     - uses: actions/setup-node@v1
       with:
         node-version: '12'
     - uses: actions/checkout@v2
       with:
         repository: env0/env0-client-integrations
     - name: install
       working-directory: node
       run: yarn
     - name: deploy
       working-directory: node
       run: >
         node env0-deploy-cli.js
         --apiKey ${{ secrets.ENV0_API_KEY_ID }}
         --apiSecret ${{ secrets.ENV0_API_KEY_SECRET }}
         --action $ACTION
         --organizationId ${{ secrets.ENV0_ORG_ID }}
         --projectId ${{ secrets.ENV0_PROJECT_ID }}
         --blueprintId ${{ secrets.ENV0_BLUEPRINT_ID }}
         --environmentName "${{ github.head_ref }}"
         --revision "${{ github.head_ref }}"

In the code above, you can see we

• Determine the action depending on the Github event data
• Fetch the env zero CLI, using the `checkout` action
• Run the env zero cli to deploy, update, or destroy the environment
• The name of the environment will be the branch name

That’s all folks!

We now have a fully functioning pipeline, and our setup will automatically create a new environment for every PR we open! When we deploy a new feature, even if that feature requires new or different infrastructure, the changes in our Terraform code will automatically be reflected in the resources provisioned for the PR environment!

In the case of env0, even though environments will be automatically created and destroyed by our CI/CD integration, we can also use the env zero UI as a control plane, for understanding which environments are up, and what they consist of. You can also use env0’s cost monitoring features, to understand how much each of these environments actually costs.

‍

Thank you for taking the time to read this post, I hope it helps you setting an environment-per-PR pipeline for your team. Once you’ve tried it yourself, I’d love to hear about it! Let me know in the comments below or on Twitter at @envzero.

Imagine managing your cloud infrastructure using the programming languages you already love—Python, Go, JavaScript, you name it. No more wrestling with YAML, JSON, or HCL (HashiCorp Configuration Language) files!

Pulumi gives you that power, offering a robust CLI and service backend to manage both state and secrets. It's like the Swiss Army knife for cloud infrastructure, supporting all the major providers like AWS, Azure, and Google Cloud.

Today we're diving into the world of Pulumi and its integration with env0. We'll explore what Pulumi is, its features, how to set it up, and even throw in a real-world example (provisioning an EKS cluster). Also, we’ll weigh the pros against the cons and look at how it stacks up against other options. So buckle up; this is going to be a fun ride!

Video Walk-through

‍

Requirements:

• A GitHub account
• An AWS account
• An env zero account
• A Pulumi account

TL;DR: You can find the main repo here.

What is Pulumi?

Pulumi is an open source Infrastructure-as-Code (IaC) framework that provisions resources utilizing common programming languages. Pulumi also supports the major cloud providers: AWS, Azure, and Google Cloud. Its leaning on common languages eliminates the time it would otherwise take to get used to a new domain-specific language like HCL. 

If you're wondering how it stacks up against Terraform, check out my previous blog comparing Pulumi vs. Terraform. But the main benefits come in three main cores: the Pulumi SDK(s), the service backend, and finally the automation API.

Pulumi SDKs

First up, the SDKs. Pulumi's SDKs are what make it super versatile. These SDKs allow you to use languages like Python, JavaScript, TypeScript, Go, or .NET for defining and deploying your infrastructure. This is super cool because it means you can use the same language you're already comfortable with for your application development. 

That all means that you end up with the following advantages: strong familiarity with the core languages, a long list of library resources (according to language), and reusable custom abstractions.

Pulumi Service Backend

Pulumi's SaaS offering comes replete with CI/CD integrations, Policy-as-Code, role-based access, and state management. 

State Management  – Safely stores and manages the state of your infrastructure. This means less headache worrying about where your infrastructure's "truth" lives. There is also an option for self-managed state through your own cloud account on AWS, Azure, or GCP.

Collaboration Features  – You can collaborate with your team on infrastructure updates, with features like RBAC, stacks history, and more.

Policy-as-Code – Enforce security, compliance, and best practices across your infrastructure using Pulumi’s Policy as Code offering called CrossGuard.

CI/CD Integration  – Pulumi CI/CD integrations work with popular systems like GitHub Actions, GitLab CI, Jenkins, TravicCI, AWS Code Services, Azure DevOps, and more.

Automation API

This Automation API can embed Pulumi directly into your application code, offering a hassle-free way to manage infrastructure. 

In essence, this concept encapsulates the core functionalities offered by the Pulumi Command Line Interface (CLI), such as executing commands like [.code]pulumi up[.code], [.code]pulumi preview[.code], [.code]pulumi destroy[.code], and [.code]pulumi stack init[.code]. 

However, it extends beyond this by offering enhanced flexibility and control. This approach is designed to be strongly typed and secure, facilitating the use of Pulumi within embedded environments, for instance, within web servers.

Importantly, this method eliminates the need for running the CLI through a shell process, streamlining operations, and integrating infrastructure management more seamlessly into application environments.

Pulumi Features

Alright, let’s dig into some of the Pulumi concepts and features that it offers:

1. Component Resources

Pulumi lets you define reusable building blocks known as "component resources." These are like your typical cloud resources but bundled with additional logic. If you are familiar with Terraform, these would be your modules.

2. Stack References

Manage dependencies between multiple Pulumi stacks effortlessly. This feature is a real game-changer for managing infrastructure at scale.

3. Templates and Packages

Think of these as the ultimate cheat codes for your IaC. Instead of starting from scratch, you can kick things off with a pre-baked setup. Here’s why they're great:

• Speedy Setup: No more blank-slate syndrome. You’ve got a starting point that’s not just a blank file – it’s a springboard that gets you coding your infra in record time.
• Best Practices: These templates aren't just thrown together – they're crafted with best practices in mind. So you're not just starting faster, you're starting smarter.
• Learning Resources: New to Pulumi or a particular cloud service? Templates can be great learning tools, showing you the ropes of how things are structured and pieced together.

How to Install Pulumi

Alright, time to get our hands dirty. Installing Pulumi is a breeze. You can reference this from Pulumi's documentation.

Since I'm running this in my Windows for Subsystem Linux environment, I can run the install script as shown:

curl -fsSL https://get.pulumi.com | sh -s -- --version 3.91.1

Pulumi Stack Example

Let's get into the meat and potatoes: stacks. A Pulumi stack is essentially an isolated, independently configurable instance of a Pulumi program. Let's first work with the Pulumi CLI then later we'll see how to use env0.

Create a New Pulumi Project

First, create a Pulumi project by creating a new directory and running the [.code]pulumi new[.code] command with the [.code]kubernetes-aws-python[.code] Pulumi template.

mkdir Pulumi-EKS
cd Pulumi-EKS
pulumi new kubernetes-aws-python

Continue by providing a project name, description, and stack name along with the AWS region and some other parameters.

Pulumi installs the necessary dependencies and your new project is ready.

Run Pulumi

Next, make sure you export your AWS cloud credentials as environment variables and run [.code]pulumi up[.code].

export AWS_ACCESS_KEY_ID=your-access-key-id
export AWS_SECRET_ACCESS_KEY=your-secret-access-key
pulumi up

Read what Pulumi is about to do, then answer [.code]yes[.code] when asked if you want to perform this update.

Now, Pulumi will start to provision resources and you will see the resources get created in the terminal as shown below.

Observe the Output Results

If all goes well, you should have your new EKS cluster up and running. You can also check the Pulumi UI for your new stack where you can view all the resources created along with the output.

You can view the output in the UI or the CLI for the vpcId and the kubeconfig.

Access the EKS Cluster

To get the kubeconfig for the EKS cluster, run the following command:

echo $(pulumi stack output kubeconfig) > mykubeconfig
export KUBECONFIG=./mykubeconfig

Now run [.code]kubectl[.code] commands to interact with the EKS cluster:

kubectl get nodes

Congratulations! You've successfully provisioned an EKS cluster in AWS.

Examine the Infrastructure Code

Take a look at the actual code that provisions our EKS cluster. Notice how it's written in simple Python. I could have built the cluster from scratch by calling on each resource, but why reinvent the wheel? There is an excellent Pulumi package called Amazon EKS in the Pulumi Registry. I decided to go with this.

As you see, in under 40 lines of code, we have our EKS cluster defined:.

import pulumi
import pulumi_awsx as awsx
import pulumi_eks as eks

# Get some values from the Pulumi configuration (or use defaults)
config = pulumi.Config()
min_cluster_size = config.get_float("minClusterSize", 3)
max_cluster_size = config.get_float("maxClusterSize", 6)
desired_cluster_size = config.get_float("desiredClusterSize", 3)
eks_node_instance_type = config.get("eksNodeInstanceType", "t3.medium")
vpc_network_cidr = config.get("vpcNetworkCidr", "10.0.0.0/16")

# Create a VPC for the EKS cluster
eks_vpc = awsx.ec2.Vpc("eks-vpc",
    enable_dns_hostnames=True,
    cidr_block=vpc_network_cidr)

# Create the EKS cluster
eks_cluster = eks.Cluster("eks-cluster",
    # Put the cluster in the new VPC created earlier
    vpc_id=eks_vpc.vpc_id,
    # Public subnets will be used for load balancers
    public_subnet_ids=eks_vpc.public_subnet_ids,
    # Private subnets will be used for cluster nodes
    private_subnet_ids=eks_vpc.private_subnet_ids,
    # Change configuration values to change any of the following settings
    instance_type=eks_node_instance_type,
    desired_capacity=desired_cluster_size,
    min_size=min_cluster_size,
    max_size=max_cluster_size,
    # Do not give worker nodes a public IP address
    node_associate_public_ip_address=False,
    # Change these values for a private cluster (VPN access required)
    endpoint_private_access=False,
    endpoint_public_access=True
    )

# Export values to use elsewhere
pulumi.export("kubeconfig", eks_cluster.kubeconfig)
pulumi.export("vpcId", eks_vpc.vpc_id)

Pulumi makes it very easy to choose between many languages right in the documentation.

If you need to tweak the cluster configuration, it's easy to do so with the very well-documented eks.Cluster package.

Pulumi Configuration Files

When we ran the [.code]pulumi new kubernetes-aws-python[.code] command, Pulumi 1) created a new folder for us, 2) downloaded dependencies in a virtual environment for Python, and 3) also created two config files. 

Let's take a look at them now.

1. pulumi.yaml

This file acts as the manifest for your Pulumi project. It's a key part of the project configuration and provides metadata about the project itself.

name: my-pulumi-eks-env0
runtime:
  name: python
  options:
    virtualenv: venv
description: A Python program to deploy a Kubernetes cluster on AWS

‍

Here's what each part of the content you've provided does:

• name  – This is the name of your Pulumi project. When you run [.code]pulumi new[.code], it sets this name, and it's used as a default prefix for the resources Pulumi creates.
• runtime  – This specifies the runtime environment that your Pulumi program is expected to run in. In your case, it's set to python, meaning the Pulumi CLI expects your Infrastructure-as-Code to be written in Python.
• options  – These are additional settings related to the runtime environment.
• virtualenv –  This option tells Pulumi to use a Python virtual environment located in the venv directory within your project directory. This is important for Python-based projects to ensure dependencies are isolated from other Python projects on the same system.
• description  – This provides a human-readable description of what the Pulumi project does. It's a string that helps you and others understand the project's purpose at a glance.

So, when you initialize a new Pulumi stack or when Pulumi interacts with your project, it uses this file to understand the project structure, runtime requirements, and other metadata that influence how it deploys and manages your infrastructure resources.

2. pulumi.dev.yaml

When you run the pulumi new command and answer the setup wizard's questions, Pulumi automatically saves these answers as configurations in the pulumi.dev.yaml file. This file acts as a record of the initial setup parameters you specified for your project.

Now, if you enter commands or make changes at a different time (i.e., not during the initial Pulumi new setup) these changes won't automatically update the pulumi.dev.yaml file. Instead, you have two main alternatives for updating configurations after the initial setup:

1. Manual Editing – You can directly edit the pulumi.dev.yaml file to change or add configurations. This is like tweaking the settings of your project by hand.

2. Using Pulumi CLI Commands – You can use specific Pulumi CLI commands to update your configuration. For example, if you want to change the AWS region, you could use a command like [.code]pulumi config set aws:region us-west-2[.code]. This command updates the configuration in your pulumi.dev.yaml file without you having to manually edit the file.

Here is the content of the file:

config:
  aws:region: us-east-1
  my-pulumi-eks-env0:desiredClusterSize: "2"
  my-pulumi-eks-env0:eksNodeInstanceType: t2.small
  my-pulumi-eks-env0:maxClusterSize: "3"
  my-pulumi-eks-env0:minClusterSize: "1"
  my-pulumi-eks-env0:vpcNetworkCidr: 10.0.0.0/16

To clean up simply run [.code]pulumi destroy[.code].‍

Pros and Cons of Using Pulumi

Pros

1. Language Choice – Use your favorite programming language.
2. Rich Ecosystem – Supports a ton of cloud providers.
3. Dynamic Providers – Extend its capabilities as you see fit.

Cons

1. Language Overload – Sometimes, choosing a language can be a burden.
2. Learning Curve – If you're coming from dedicated DSL tools like Terraform's HCL, there might be an initial hump.

Pulumi Alternatives

The most obvious alternative to Pulumi is Terraform. But hey, keep an eye out for OpenTofu, an upcoming open-source alternative following a BSL license change. Crossplane is another alternative for those who enjoy building infrastructure using Kubernetes CRDs. Check out more details below.

1. Terraform

Overview: Terraform is a big player in the IaC field. It uses its own domain-specific language, HCL (HashiCorp Configuration Language), which is designed to describe infrastructure in a declarative way.

Why It's Popular: Terraform's been around for a while and has a huge community and support base. Plus, it works across many cloud providers, making it super versatile.

Key Differences from Pulumi: Unlike Pulumi, Terraform isn’t based on conventional programming languages. So, if you're not into learning HCL, it might be a bit of a curve.

2. Crossplane

Crossplane is perfect for those who are all-in with Kubernetes. It allows you to manage your infrastructure using Kubernetes CRDs (Custom Resource Definitions).

If you’re comfortable with Kubernetes and want to manage cloud resources as Kubernetes objects, Crossplane is your go-to. Being Kubernetes-focused, it fits well in ecosystems already heavy with Kubernetes usage and has a growing community.

Thoughts

Each of these alternatives has its own flavor. Terraform is the established giant with a dedicated language, OpenTofu promises to always be open-source along with new approaches to IaC, and Crossplane merges the worlds of Kubernetes and IaC.

Depending on your needs, comfort with certain technologies, and the specifics of your infrastructure, one of these might be a better fit for you than Pulumi.

Tutorial: Using Pulumi with env0

Now let's see how to use env zero to create the same Pulumi stack. We will create the same EKS cluster but this time by using env zero to trigger Pulumi.

Let's start by creating a new project in env0.

Next, you'll need to create a Pulumi template as shown:

Then connect to your VCS. Make sure to select the Pulumi folder, in our case Pulumi-EKS.

Under variables, add your PULUMI_ACCESS_TOKEN environment variable.

Then finally, make sure this template is deployable in our 'eks-demo' project.

AWS Cloud Provider Credentials

Make sure you have your AWS credentials set up in the Project settings

Create an Environment

Now we're ready to create a new environment. Head over to 'Project Environments' then create a new e.

When you see the eks-template, click the 'Run Now' button. There are some options to use such as enabling drift detection and the ability to automatically destroy the environment. When you're ready click the Run button.

Notice in the deployment logs how we have a 'Before: Pulumi Preview' step. This is defined in the env0.yaml file at the root of our repo to provide our configuration variables.

Below you can see how our env0.yaml looks like. Notice that we are specifying the same configuration variables that were in our pulumi.dev.yaml file.

version: 1
deploy:
  steps:
    pulumiPreview:
      before:
        - cd Pulumi-EKS && pulumi config set-all \
          --plaintext aws:region=us-east-1 \
          --plaintext my-pulumi-eks-env0:desiredClusterSize="2" \
          --plaintext my-pulumi-eks-env0:eksNodeInstanceType=t2.small \
          --plaintext my-pulumi-eks-env0:maxClusterSize="3" \
          --plaintext my-pulumi-eks-env0:minClusterSize="1" \
          --plaintext my-pulumi-eks-env0:vpcNetworkCidr=10.0.0.0/16

If you left the option to approve the plan automatically unchecked, you will need to confirm the execution of the [,code]pulumi up[.code] command.

View the Output

Finally, once the deployment completes, you can view the outputs under the 'Resources' tab.

Once again, to access the Kubernetes cluster, you can simply save the kubeconfig in a file and export as an environment variable as shown below:

export KUBECONFIG=./mykubeconfig
kubectl get nodes
NAME                           STATUS   ROLES    AGE   VERSION
ip-10-0-144-143.ec2.internal   Ready       73m   v1.28.2-eks-a5df82a
ip-10-0-29-122.ec2.internal    Ready       73m   v1.28.2-eks-a5df82a

Congratulations! You've just used env zero to deploy the Pulumi stack and provision an EKS cluster, and it probably took less than 5 minutes.

To clean up, just click the 'Destroy' button. One click and it gone.

In Summary

We've covered a lot of ground in this post—from the nuts and bolts of what Pulumi is to its nifty features, and even how it plays nice with env0.

If you're in the DevOps or Platform Engineering space, Pulumi offers a refreshing take on infrastructure-as-code. By marrying traditional programming languages with cloud resources, you get a level of flexibility and power that’s hard to beat.

So, what's the takeaway? If you’re looking to step up your infrastructure game, Pulumi is worth a shot.

Not ready for your entire team to move from Terraform to Pulumi? That's the benefit of a framework agnostic IaC platform such as env0.

Here are some of the key features that I like about env0:

• Drift detection – env zero provides drift detection that can help you detect drifts and alert you about them automatically.
• Governance – our platform allows you to define custom policies and guardrails to both secure and keep your infrastructure compliant.
• Multiple frameworks – env zero supports multiple frameworks such as Pulumi, Terraform, OpenTofu, and more.
• Ephemeral environments – Developers can set up an environment with a timer to self-destruct reducing wasted resources.
• Flexibility – With pre- and post-hooks that reduce the need for a full external  CI/CD pipeline.

For more information on env0's support of Pulumi, please reference this guide.

‍

Today, we’re excited to announce that env zero is open sourcing Terratag - a CLI tool that enables users of Terraform to automatically create and maintain tagging across their entire set of AWS, Azure, and GCP resources. It enables you to easily add dynamic tags to your existing Infrastructure-as-Code and benefit from some of the cross-resource tag applications you wish you had thought of when you had just started writing your Terraform, saving you tons of time and making future updates easy.

Tagging is every cloud provider’s way of enabling you to organize and manage your cloud resources - for everything from automations to cost insights, tagging enables it all. But the actual process of tagging today is broken, requiring too much manual work with too many mistakes during it. That’s why we built Terratag to automate it all. We wanted to solve this task not just for env zero users, but for the whole developer community. Our hope is that making Terratag available for everyone will help more organizations adopt infrastructure-as-code for software development.

Why is Tagging Important?

All major cloud providers allow for tagging (or labeling) cloud resources. Moreover, they all encourage you to do so in order to benefit from their complementary services; allowing you to manage budgets, set up powerful automation algorithms, and unlock insights offered both by the cloud providers themselves, and independent third parties.

Harnessing powerful infrastructure-as-code frameworks such as Terraform to define and tag your cloud resources allows for useful dynamic tag applications on various verticals. Nevertheless, it’s important to use the right tag for the right job. Some of the most common tags include:

• Technical Tags can be used for versioning your cloud resources or specify Environment or application ID to be able to easily filter or group together resources under the umbrella of a single environment.
• Tags for Automation such as date/time tags that denote a resource should be started, stopped, deleted, or rotated.
• Business Tags can group together resources based on their business need - in a single tenant or dedicated environment it is very useful to tag a group of cloud resources with the customer ID, project, owner or cost center.

Here’s an example of how one would tag a cloud resource using Terraform:  

-resource "aws_vpc" "example" {
 # ... other configuration ...  
 tags = {    
   Name        = "MyVPC"    
   Owner       = "Operaions"            
   Environment = "${var.environment}"        
   Project     = "${var.project}"  
 }
}

Pretty powerful, yet simple, right?

And things tend to remain simple, at first.

The Problem with Tagging Today

Having only a few cloud resources makes it very easy to add both static and dynamic tags to these resources. However, as your infrastructure grows, having to repeat those same tags over and over for tens or hundreds of cloud resources can become a hassle to maintain. But that’s just the tip of the iceberg. Manual tagging processes fail on other important ways:

• Standards are hard to maintain if not enforced - Your entire team needs to be onboard and keep an eye out for newly added cloud resources, making sure they include those tags or you may miss some significant resources when acting on that metadata later.
• Harder to change - Applying changes to tag structure across the board quickly becomes a rather unmanageable task.
• Metadata can obscure what’s important - While all this tagging metadata is extremely useful for slicing and dicing later, having it everywhere on your resources is polluting your lovely IaC - making it much more verbose and harder to maintain.
• New to tagging - What if you already have plenty of Terraform modules with cloud resources which weren’t tagged to begin with? Trying to tag them all now can be painstaking work.

Infrastructure-as-code is, well, just code. And such is the case with any code - code repetition makes it harder to fix errors, apply enhancements, make adjustments and maintain readability.

Lack of proper layering or aspect control makes it harder to retrofit existing solutions.
A cross-cutting concern calls for a cross-cutting solution.

How Terratag Works

Terratag is a CLI tool allowing for tags or labels to be applied across an entire set of targeted Terraform files directory.

It generates Terraform files with your custom tags added to any GCP, AWS or Azure cloud resources:

$ terratag -dir=ops -tags={\"environment\": \"$ENVIRONMENT\",\"project\": \"$PROJECT\"}

Using Terratag as a step before terraform planning or terraform applying allows you to dynamically inject the powerful metadata of tags or labels across a set of resources - freeing you and your team from the burden of remembering to constantly apply cross-cutting tags to new resources or maintain and modify tags on existing resources.

Now you can also easily add dynamic tags to your existing IaC and benefit from some of the cross-resource tag applications you wish you had thought of when you had just started writing your Terraform code.

How do we use Terratag in env0?

env0 is a management platform that lets your team run their own cloud environments, governed by your policies and with complete visibility & cost management. 

We use Terratag to drive our Cost Management and Resource modules. We automatically inject dynamic tags for your managed Environments and Projects - allowing us to provide you with powerful insights such as viewing your entire multi-cloud cost per environment to date, over time, and much, much more.

With env0, there’s no need to deal with your own management system or build Terratag into your setup, we handle it all for you automatically.

Terratag is 100% open source

At env0, we love open source, which is why we’re making Terratag available to all.

Check out our Terratag repo on GitHub. Issues, suggestions, requests and of course, Pull Requests, are very welcome!

Web applications, Single page applications, static websites. They are everywhere. No matter if your back-end is running on Kubernetes or serverless, public cloud or on-premise, if you have a front-end, there is a good chance it is a browser rendered, statically delivered bundle of HTML, CSS and Javascript. 

In my previous blog posts, I discussed ‘Feature Environments’ and ‘Per-Pull Request Environments’. Making your entire system run such environments is no small feat, especially if you are running complex infrastructure, a very large scale system, or legacy code without IaC. But you can still make a small effort and gain a lot of value, by adapting these isolated environments for your front-end. It’s the perfect place to start, because

• Statically hosted front-ends tend to live on relatively simple and stateless infrastructure.
• Sharing a feature-in-progress on the front-end is the most interesting part of the feature to share, and non-technical stakeholders will want to see that, e.g., a Product Manager that wants to do acceptance tests. 
  

Per-pull request environments for the front-end are commonly called “Preview environments” and can be achieved in many ways. Some CI/CD providers even offer preview environments as a paid feature, but I’d like to show you how easy it is to do this on your own - using Terraform. Rolling this out on your own, will give you better control over your infrastructure, allowing you to create preview environments which are closer to your production environment, and will probably cost you less - by cutting out the middleman.

To demonstrate how, I’ll be running a simplified React app, packaged with Parcel, and using Terraform for resource provisioning, Github as a VCS, and env0 as my environment management platform. The full source code can be found here.

There are two important files we need to look at - the first one is `env0.yaml` :

version: 1
deploy:
 steps:
  terraformInit:
   before:
    - yarn 
    - yarn run build 
  terraformOutput:
   after: 
   - aws s3 sync ./dist s3://$(terraform output s3_bucket_name)

env zero will handle our Terraform files natively, taking care of creating, updating, and deleting any cloud resources we need. Before env zero initializes Terraform, we want to build our frontend. We want to do that before, so in case anything fails, we won’t provision the infrastructure for nothing.

After Terraform completes, we will run an aws CLI command, to sync our dist folder to our s3 bucket. This will upload our built website files to the bucket for hosting.

The second file we should notice is our `pr-environments.yaml`, located in the `.github` folder. It will tell Github to trigger env0, everytime we open, update, or close a Pull Request.

name: "PR Environments"
on: 
pull_request: 
 types: [opened, closed, reopened, synchronize]
jobs:
 env0_pr_environment:
  name: "PR Environment"
  runs-on: ubuntu-16.04
  container: node:12  
  steps:     
   - name: Install env zero CLI
    run: yarn global add @env0/cli     
   - name: Deploy Environment     
     if: github.event.action != 'closed'     
     run: >         
      env zero deploy   
      --apiKey ${{ secrets.ENV0_API_KEY_ID }}   
      --apiSecret ${{ secrets.ENV0_API_KEY_SECRET }}   
      --organizationId ${{ secrets.ENV0_ORG_ID }}     
      --projectId ${{ secrets.ENV0_PROJECT_ID }}     
      --blueprintId ${{ secrets.ENV0_BLUEPRINT_ID }}   
      --environmentName "${{ github.head_ref }}"  
      --revision "${{ github.head_ref }}"   
   - name: Destroy Environment   
     if: github.event.action == 'closed'   
     run: >  
      env zero destroy    
      --apiKey ${{ secrets.ENV0_API_KEY_ID }}    
      --apiSecret ${{ secrets.ENV0_API_KEY_SECRET }}   
      --organizationId ${{ secrets.ENV0_ORG_ID }}  
      --projectId ${{ secrets.ENV0_PROJECT_ID }}   
      --environmentName "${{ github.head_ref }}"

The steps here are pretty simple - we install the env zero CLI using a simple `yarn install` command, and we run the env zero CLI with the required parameters. The CLI will already know to create or update an environment based on the name, which will be the same as the branch we are working on. 

When the PR will be closed, we will run the CLI with the `destroy` command, and env zero will take care of cleaning up our cloud resources, so we can keep a clean cloud account, with low costs. 

Looking at an open Pull Request, Github informs us that it ran the deployment successfully - 

‍

Looking at env0, we can see a list of all our PR environments, and for each one, we can use the “Outputs” section, to get the environment’s website URL - 

That way I can Slack my product manager, and let them know they can already play around with the new video feature we’ve been working on, so I can get feedback early!

Advanced Environment Management‍

Using env zero for managing preview environments opens up some new options that are only available on an advanced environment-as-a-service platform such as env0. To name a few:

• Time To Live for environments - making sure I don’t have any leftover resources.
• Detailed cost monitoring - getting actual costs per environment is a huge help in understanding where our cloud costs come from.
• Role based access - anyone in the organization can access env0, and I know they won’t do any damage, when they are assigned the correct role. 

Thank you for taking the time to read this post, I hope it helps you setting up preview environments for your web application. Once you’ve tried it yourself, I’d love to hear about it! Let me know on Twitter at @envzero.