Export env zero Log Data to SIEM and Monitoring Platforms

Cost visibility without governance is just expensive awareness. Governance without real-time data is just expensive guessing.

The Infracost team published something worth reading: a breakdown of the ten cloud cost anti-patterns that quietly drain infrastructure budgets. These include old EC2 instance families, orphaned test environments, log groups retaining data forever, and multi-AZ replication running in dev because someone copy-pasted a prod config.

Their diagnosis is correct. These aren't careless mistakes. They're reasonable decisions made without cost context at the moment those decisions happen.

But the billing console only tells you what already happened. And a PR-level cost estimate only tells you what's about to happen. Neither one tells you what your infrastructure is costing you right now, across every environment, every team, every account.

That's three separate problems. They need three separate capabilities… working together.

The Three Moments That Matter

Cloud cost control has a timeline, but most teams only address one part of it.

Before deployment: An engineer writes a Terraform module. They pick an instance type, configure a database, set retention policies. Every decision they make has a cost implication. Without visibility at this stage, those implications are invisible until the bill arrives.

At deployment: Code merges. Infrastructure provisions. Policies either enforce standards or they don't. This is the last moment a cost problem is cheap to fix.

After deployment: The environment is live and then configurations drift. New services get adopted without a clear cost model, while test environments survive longer than they should. Costs accumulate quietly, outside any PR, outside any policy review.

Most teams have partial coverage at best. Infracost handles the first moment. env zero handles the second. CloudQuery—now part of env zero—handles the third.

Before You Ship: Infracost + env zero

Infracost surfaces cost estimates in pull requests (and they support more than 10 million SKUs across clouds). An engineer sees that the RDS instance is over-provisioned for a dev workload. They see the monthly delta. The context is there, at the exact moment the fix is a two-minute edit rather than a production change requiring a maintenance window.

env zero extends that context into enforcement. A cost threshold policy blocks the deployment if the estimated spend exceeds a defined limit. A tagging policy requires every environment to carry an owner tag and an expiry date before it merges. A governance workflow routes high-cost changes to a reviewer with authority to approve or reject.

The engineer isn't the last line of defense anymore; the platform is.

Customers using env zero's pre-deployment cost enforcement have cut IaC-related infrastructure spend by 45%. That's not optimization after the fact; that's the problem not happening.

After You Ship: env zero + CloudQuery

Pre-deployment governance catches what you can see before the merge. It doesn't catch what changes after it.

Infrastructure drifts. Engineers provision resources outside the standard pipeline. A test environment spun up six months ago never got an expiry date. A new managed service got adopted without anyone mapping out what it costs at scale. These problems live in your running infrastructure, not in your pull requests.

CloudQuery is now part of env zero. It continuously queries your live cloud infrastructure—across AWS, Azure, and GCP—and surfaces what's actually running, what it's actually costing, and where it deviates from the standards your platform team has defined.

This is the real-time half of cloud cost governance. Not what an engineer estimated before the merge. What your infrastructure is actually spending right now.

The Test Environment Problem, Fully Solved

The Infracost piece calls out the test environment that never got torn down. Every platform engineer has several of these.

env zero handles this at the policy level before deployment: every environment carries a TTL, auto-expiry terminates it on schedule, no engineer has to remember anything.

CloudQuery handles the environments that already exist without those controls. It finds them. It surfaces their running cost. It gives your platform team the data to act on.

Virgin Media O2 went from spending half a day standing up a single POC environment to spinning one up in under 10 minutes with 5 variables. Auto-expiry controls costs automatically. CloudQuery makes sure nothing slips through the cracks of what's already running.

Governance Across the Full Infrastructure Lifecycle

The common thread in every cloud cost anti-pattern Infracost identified: engineers made reasonable decisions without the right context at the right moment.

The fix isn't better finance reviews after the fact, but rather, closing all three gaps before deployment, at deployment, and in production.

Infracost provides cost context in the PR. env zero enforces the policies that make that context binding. CloudQuery, now part of env zero, continuously monitors what's running and surfaces what the enforcement layer didn't catch.

That's not three tools. That's one governance lifecycle from the first line of Terraform to the last running resource.

See how env zero governs the full infrastructure lifecycle →

Learn more about the env zero Infracost integration →

Learn more about CloudQuery →

Related: env0’s AWS integration brings environment management, cost controls, and deployment guardrails to your AWS infrastructure workflows.

Hello, env zero fans! As some of you know, we have almost unlimited extensibility with 3rd party tools, using our custom workflows. You can hook in pretty much any tool, in any phase of the deployment. Today, we’re going to talk about how to prevent cloud misconfigurations before they start. We’re going to do this by chaining a tool in the deployment after the terraform plan phase. This is where our friends at Bridgecrew come in. Just like we at env zero have open-sourced the Terratag module of our platform, Bridgecrew has open-sourced Checkov!

Checkov

Checkov is a static code analysis tool for infrastructure-as-code. It scans cloud infrastructure managed in Terraform, Cloudformation, Kubernetes, Arm templates, or Serverless Framework and detects misconfigurations.

Setup

For illustration purposes, we’re going to use Bridgecrew’s demo application called TerraGoat. TerraGoat is Bridgecrew’s “Vulnerable by Design” Terraform repository. TerraGoat is a learning and training project that demonstrates how common configuration errors can find their way into production cloud environments.

DISCLAIMER: DO NOT ACTUALLY DEPLOY THIS APPLICATION INTO YOUR CLOUD INFRASTRUCTURE. IT IS PURPOSELY COMPROMISED.

I have created a template of TerraGoat inside of env zero and linked it to our Bridgecrew Demo project.

The only other thing we have to do is to actually call Checkov to do the check during the deployment. We need to do this after the Terraform plan phase, so that we have a plan to check. Here is what the env0.yml file will look like:

This adds 3 commands that run after the Terraform Plan, and before Terraform Apply. We put it here so that the Apply doesn’t run in case of failures. We don’t want to see the errors after the resources are applied. We want the deployment to fail if there are errors.

This command installs Checkov into our runtime environment using the pip3 package installer so we can run it against our Terraform plan.

This command essentially formats our .tf-plan file into tf.json so that it can be parsed and run against Checkov.

This command has a lot going on and is in 2 parts. First, it quietly executes Checkov against our tf.json (the reformatted tf.plan file) and looks for a 0 exit code. The double pipe || tells bash to only execute the 2nd command if the exit code of the first command is not 0. So if your Checkov results are clear, your deployment gets the 0 exit code and continues on with the deployment.

If not, then the second part of the command runs. Knowing if this part runs, it is because of a failure, we’re just going to format our error message here. We run Checkov again so we can pipe the error with the echoed error notification text to the console. The 1>&2 routs stdout to stderror, and the exit 1 code tells env zero that the stage failed, and to end the deployment run.

The env zero platform will parse the error, and give you the clear error printed on the Environment deployment page. But, if you want the full logs from Checkov, you can find those in the After: Terraform Plan deployment logs.

And that’s it! A little bit of YAML, and you’ve implemented Checkov to protect yourself against the deployment of misconfigured cloud resources. That is instantly added value to your organization by shifting the security left in your deployment process with env0.

You can find more information on Checkov here. You can find the open-source repository on GitHub. And be sure to see how you can automate your infrastructure security from commit to cloud at Bridgecrew.io.

‍

This quick video talks about the static code analysis tool, Checkov by Bridgecrew. We'll use it to fix a problem with a Terraform deployment into Azure. Be sure to star their repository if you like the project.

“All these investments in DevOps and developer productivity and operational efficiency are bottlenecking in these environments on security and compliance.”

This was an observation from Andrew Clay Shafer in a fireside chat with John Willis at DevOps Days Dallas 2022. One such bottleneck can come in the form of audit logs, but auditability doesn’t have to be a blocker. 

In this post we’ll take a look at why auditing is necessary for Infrastructure as Code, the benefits of having an audit trail for Terraform and other IaC frameworks (such as Terragrunt, Pulumi, CloudFormation, Kubernetes, and others), and share how you can automate your audit trail easily with env0.

How do you audit Infrastructure as Code? 

All you need is a central log management system that can aggregate and parse your audit. env zero delivers a number of benefits for auditability:

• Changes to infrastructure are tracked in the code repository, making it easy to see who made what changes and when.
• Automatically roll back changes that cause problems.
• Audit changes made by third-party vendors.

Audit log walkthrough 

There are two ways to access audit logs in env0, 1) through the UI, and 2) through the API. As always, env zero gives you the flexibility to interact with your infrastructure-as-code as it suits your organization, be it programmatically or through a simple user interface. Only an Admin user can access the Audit Logs.

The audit log contains information about who performed the activity, when the action was performed, the activity’s description, and additional data like their IP address.

The audit log shows events related to changes in your:

• Organization
• Projects
• Templates
• Environments
• Teams
• Users
• Roles
• Modules
• Git Tokens
• Cloud credentials
• API keys
• SSH keys
• Variables
• Agent configurations

Through the env zero UI

1. Go to the Organization's Settings page.
2. Click the Audit Logs tab.
3. The audit details are listed in a table.
4. Click the row's + sign to reveal additional activity details.
5. Click the Show more button at the bottom of the page to see more rows.

Through the env zero API

Use the "Fetch Audit Logs" API to retrieve your organization's audit logs programmatically. Learn more about the audit log in our documentation.

Use an audit log for governance and compliance

With this latest release, we’re continuing to deliver the enterprise-level capabilities to enable DevOps, Platform, and Infrastructure as Code teams to deliver secure software faster while ensuring compliance with internal policies or external regulations. env zero has clearly auditable versioning of infrastructure changes, giving you the answer to questions like, “What was changed, who changed it, and when was it changed?” 

We help you avoid the audit and compliance bottleneck, so you can focus on what’s important—shipping software.

In recent years, Infrastructure as Code (IaC) has become an increasingly popular approach for managing cloud-based infrastructure with tools such as Terraform, AWS CloudFormation, and Pulumi.

Those tools allow developers to define infrastructure resources using code that enables them to use all the capabilities of code, such as versioning, automation deployment, and policies. However, with the growing complexity of cloud architecture, it is challenging to monitor and debug issues in real-time, such as CPU spikes or number of requests.

Observability is the ability to understand how a system behaves internally based on its external outputs. In other words, it's the practice of instrumenting a system to expose its internal state in a way that is observable and understandable. Observability is essential for any system to detect and resolve issues quickly, where downtime can be costly.

For example, you've defined your infrastructure resources and deployed them using a tool like Terraform. Terraform will let you know all resources have been deployed successfully, but there are no checks to validate the health of those resources over time. 

If, after time, you notice that the application response time is increasing, and customers are complaining about slow performance and bad user experience. With an observability tool, you can quickly identify the issue by inspecting application logs and traces and correlating them to your infrastructure changes.

You may discover that one of the infrastructure resources, such as a load balancer, some of the virtual machines, or your application’s database is not scaling properly, leading to increased response time. By identifying the issue quickly, you can adjust the IaC code to make changes to the infrastructure resources and redeploy the resources with minimal downtime.

Using a full-stack observability platform with IaC provides several benefits. 

• It enables teams to detect and resolve issues quickly, reducing the impact on customers and the company. 
• It provides a holistic view of the system, enabling teams to monitor both application and infrastructure metrics in one place and understand the system in an end-to-end manner. 
• And, it allows teams to make data-driven decisions when optimizing infrastructure resources, leading to better performance and cost savings.

With env0, you can forward all env zero deployment logs to one of the supported observability platforms by going to the organization settings -> Integration -> Log Forwarding and selecting the relevant platform you are using. Doing so will allow the platform to collect logs and data from env0, and you can immediately get to observing what’s really happening under the covers of your deployed application.

In conclusion, using an observability product with IaC is essential for teams managing cloud-based infrastructure. It provides critical monitoring, tracing, and logging capabilities, enabling teams to detect and resolve issues quickly. With the growing complexity of cloud architecture, observability is becoming increasingly important, and using a product that can provide a holistic view of the system is key to optimizing performance and minimizing downtime.

‍

“We looked at a lot of tools, but env zero really stood out. It provided us with a mature enterprise-ready solution, robust integration options, and an easy way to keep our costs under control.”

Troy E. Lillehoff

VP of Cloud Strategy

Background

Western Union has been a global leader in money transfer and payment services for over 170 years, operating in more than 200 countries and serving over 150 million customers.  

The company’s mission is to “make financial services accessible to people everywhere” by building and offering easy-to-use products and services that bridge digital and physical, providing customers with choice, security, and reliability. 

Western Union's platform facilitates cross-border transactions, bill payments, and a large portfolio of other financial services. Its development department consists of over 2,000 developers managing more than 200 applications, which speaks to the scale and complexity of its worldwide operations.

Challenge

Given the vast scale of its operations, Western Union faced significant challenges in managing and scaling its cloud infrastructure. 

Specifically, Western Union's IaC deployment processes fell short of the company's high standards for cohesion and efficiency, with many issues stemming from different teams using a mix of methodologies and non-dedicated solutions such as Jenkins, Cloudbees, Spinnaker, and Morpheus, as well as different IaC frameworks – primarily, Cloudformation and Terraform.

This situation resulted in prolonged deployment times, over-reliance on manual processes, lack of broad visibility, and sub-par cost controls, leading to unpredictable IaC-related expenses and difficulties aligning with the overall FinOps strategy.

Lastly, as it was looking to modernize its IaC usage, the company also sought a way to better comply with the demanding regulations of its industry, which required meticulous attention to security, uptime, and other critical performance metrics.

Solution

To remedy the situation, Western Union embarked on a thorough search to identify an IaC management solution that would unify and optimize IaC usage across the organization. 

Equally important, they were looking for a strategic partner to assist with the implementation and scaling of their IaC operations, bridging expertise gaps in adopting new technologies.

During their due diligence, the team assessed several leading commercial solutions, including HashiCorp’s Terraform Cloud, as well as cloud-native tools like Azure DevOps. 

After careful consideration, Western Union determined that env zero was the best fit for their IaC management needs, with other options falling short in terms of visibility capabilities, integration options, ease of use, and other POC criteria.

As part of their POC summary, Western Union commended env0’s technical support for quickly resolving issues and providing a clear path to value, as well as for functioning as IaC consultants and helping adopt DevOps best practices and strategies.

In the words of Troy E. Lillehoff, Western Union’s VP of Cloud Strategy: “You taught us how to fish.”

Results

Following a successful POC, Western Union became an env zero customer and expanded the use of the platform, leading to the following outcomes:

• Enhanced Security and Compliance: The team implemented dynamic Role-Based Access Control (RBAC) and granular pre-deployment checks via Open Policy Agent (OPA). These ensured that security and compliance standards were proactively met, providing a superior alternative to post-deployment scans.
  
  
• Accelerated Deployment Times: env0’s platform, described by the team as “the easiest IaC tool for developers,” enabled engineers to deploy IaC in a self-service model without causing any disruption, thanks to the OPA-defined guardrails. Having this ‘golden path’ resolved DevOps bottlenecks and reduced deployment times from weeks to hours, and even minutes.
  
  
• Improved IaC Cost Efficiency: Leveraging env0's cost controls and FinOps integration, the team facilitated proactive cost management and aligned with the organization's FinOps strategy. This resulted in the implementation of a "chargeback" model and a significant reduction in IaC-related costs.
  
  
• Unified View of all IaC Resources: Using env zero provided various R&D teams with a shared source of truth for IaC-related activities. This facilitated collaboration on shared projects and offered an easy way to observe and analyze IaC usage across the entire organization and three different cloud providers.
  
  
• Streamlined Cross-team Processes: Using env0’s integration options, security, and FinOps teams were able to embed their policies into IaC deployment flows, ensuring effortless coordination with the R&D groups.

Next: On-Prem

Partnering with env zero helped Western Union improve security, boost deployment speeds, optimize costs, and align its IaC strategy with the latest best practices, significantly enhancing reliability and collaboration.

With the demonstrated success in the cloud, Western Union’s next steps include expanding its use of env zero to include on-premise use cases. 

The IaC automation and management solutions offered by env0, along with its developer self-service capabilities, are now a constant part of Western Union’s cloud strategy and a key driver of its future growth.

‍