OpenTF forks Terraform, insists HashiCorp is the splinter group

Blog

read time

min

‍Infrastructure-as-code: Why IaC Matters for Cloud Provisioning

Infrastructure-as-Code (IaC) allows engineers to manage cloud assets, databases, and networks as code, enhancing efficiency and consistency and bypassing manual ClickOps for cloud infrastructure provisioning and deployment.

Infrastructure code: Key Benefits of IaC

Automation and Error Reduction: Using IaC, you can automate the setup of a complex server environment in minutes with a single script, eliminating hours of manual configuration. This not only accelerates deployment but also significantly reduces the chance of human error.
Version Control and Collaboration: Enable version control to track changes and enhance team collaboration. For instance, when a team member submits a pull request for an IaC configuration change, the entire team can collaboratively review, discuss, and approve the IaC changes.
Scalability: By changing a few lines in your IaC script, you can effortlessly scale your cloud infrastructure to handle the increased load, transforming what used to be a complex process involving manual adjustments in your cloud environment.
Multi-Cloud Support: IaC tools are cloud-agnostic, meaning they offer flexibility in deploying infrastructure across various cloud environments (AWS, Azure, GCP, etc).
Security: IaC tools integrate with security frameworks like OPA, enabling engineers to define security policies and check compliance before (or after) the infrastructure provisioning process, ensuring infrastructure security.

In this post, I’ve selected a range of the most popular IaC tools to provide a quick overview of the ecosystem and help you find the IaC option best suited for your organization or project.

‍Terraform CLI IaC Tool (Terraform Code and State)

Terraform CLI is the most popular and widely used IaC tool today. Owned by HashiCorp, which was recently acquired by IBM, Terraform is designed to provision and manage infrastructure resources through code.

Terraform key features are:

Multi-cloud Support: Offers extensive multi-cloud support, enabling efficient infrastructure management across various platforms like AWS, Azure, GCP, and env zero, with the same tool and syntax.
Declarative Syntax: Terraform employs the declarative HashiCorp Configuration Language (HCL) to describe the infrastructure's desired state through IaC.
State Management: State management in Terraform uses a state file to track resources, serving as a source of truth for efficient updates, deletions, and metadata tracking.

Get Started

You should follow a series of steps to get started with Terraform:

1. Install Terraform

You can download and install Terraform on your local machine. It supports different operating systems like Windows, Linux, and MacOS.

2. Write your first Terraform Configuration

Create a new directory for your Terraform project and create a file named main.tf inside that directory. We'll take an example of writing the Terraform configuration for provisioning an S3 bucket in main.tf. Here is an example configuration:

This configuration:

Defines an AWS provider and sets the region where the resources will be created.
Defines a resource block for an AWS S3 bucket with a unique bucket name, sets the access control list (ACL) to private, and adds some descriptive tags.

3. Run Terraform Workflow (Terraform CLI in Action)

You would now need to basic Terraform commands to provision this S3 bucket in AWS in the real world.

[.code]terraform init[.code]: This command prepares your directory for Terraform operations by downloading the necessary provider plugins.

[.code]terraform plan[.code]: This command will show you a plan of all the resources Terraform will create, modify, or destroy based on your configuration. The [.code]plan[.code] output shows creating an S3 bucket (env zero-dev-bucket).

[.code]terraform apply[.code]: If you're satisfied with the plan, apply the configuration to create the S3 bucket by running [.code]terraform apply[.code]. After confirming with a ‘yes’, you’ll see the S3 bucket provisioned in your cloud environment.

‍Licensing
‍Recently, Terraform moved from Mozilla Public License (MPL) to Business Source License (BSL), confirming it will no longer be an open-source project. For instance, since HashiCorp moved from its open-source license, the Gitlab community has rooted to replace Terraform CI templates with OpenTofu templates for SaaS and Self-Managed features in GitLab, since its internal legal analysis restricts shipping Terraform CI templates.

Terraform CLI is also accompanied by Terraform Cloud (TFC), which is a HashiCorp's commercial offering. Visit here to learn what makes env zero a great Terraform Cloud alternative.

‍OpenTofu IaC Tool: Open Source Terraform Alternative

‍OpenTofu is an open-source alternative for Terraform and a drop-in replacement for Terraform v1.6. OpenTofu is forked from Terraform, and initiated by env zero, Gruntwork, Harness, Spacelift, and others as a response to HashiCorp's shift from an open-source MPL license to the Business Source License (BSL) for Terraform.OpenTofu comes with all the features of Terraform, except that it is open-source. Moreover, OpenTofu has its own registry that uses the same modules and providers that can be used with Terraform.

‍Key features:

Open Source Nature: OpenTofu's open-source nature ensures that its codebase is freely accessible to all, fostering a transparent and inclusive environment for users and developers.
Community-driven Development: OpenTofu's development is community-driven, emphasizing collaboration in feature suggestions, bug reporting, and documentation enhancements, making the process transparent and feedback-valued for users.

‍Get Started

‍You can get started with OpenTofu by just adding or replacing the Terraform binary with the OpenTofu binary, defining the configuration with the same HCL syntax, and running your workflow.

1. Install OpenTofu

Follow the OpenTofu installation guide with respect to your operating system.

2. Define your OpenTofu configuration

OpenTofu uses the same HCL syntax for defining your infrastructure code. Let’s take the same example of provisioning an S3 bucket:

3. Run OpenTofu workflow

After defining your configuration, run the same workflow as Terraform with some minor changes in syntax (replace [.code]terraform[.code] with [.code]tofu[.code] when running workflow commands).

[.code]tofu init[.code]: same as [.code]terraform init[.code] it initializes the directory by downloading provider plugins and modules.

[.code]tofu plan[.code]: Again, identical to the terraform command it shows you an execution plan to modify, create, or destroy resources. In our case, we’re creating an S3 bucket.

[.code]tofu apply[.code]: After running the command, Opentofu generates an execution plan again and asks for confirmation. Upon confirming with a ‘yes’, it creates the env zero-tofu-bucket:

‍Licensing
‍OpenTofu is under the open source Mozilla Public License (MPL) and was acquired by the Linux Foundation.

Pulumi IaC: Infrastructure-as-code Tools for Developers

Pulumi is an open-source Infrastructure as Code (IaC) tool enabling engineers to manage cloud infrastructure with languages like JavaScript, TypeScript, Python, Go, and .NET, offering flexibility beyond traditional declarative-syntax IaC tools like Terraform.

Here are some of its key features:

Multi-Cloud Support: Like Terraform, Pulumi too offers a wide range of integration with various cloud providers like AWS, GCP, Azure, etc.
State Management: Pulumi automatically manages the state of your infrastructure, keeping track of your resources and their relationships.
Secret Management: Pulumi provides secure handling of sensitive information, such as passwords or API keys, integrating with existing secrets management tools like AWS KMS, Azure Key Vault, or HashiCorp Vault.

Get Started

1. Install Pulumi and create a project

There are different ways of installing Pulumi on your local system, and you can find everything here. You can follow along on creating a Pulumi project and get started writing Python code for provisioning an S3 bucket.

2. Write a script to provision infrastructure

Here's a basic Python script to deploy an S3 bucket:

3. Deploy Stack

Run [.code]pulumi up[.code] to provision the S3 bucket (env zero-pulumi-bucket):

Licensing
Pulumi is an open-source project licensed under Apache License 2.0.

Crossplane IaC Tool for Kubernetes

Crossplane is an open-source Kubernetes add-on that makes cloud infrastructure management native to Kubernetes through custom resource definitions (CRDs), enabling provisioning and management of cloud resources directly from Kubernetes.

It allows you to define your infrastructure using Kubernetes manifests, making the entire cloud infrastructure a Kubernetes-native experience.

Key features:

Universal Control Plane: Crossplane makes Kubernetes a universal control plane with CRDs for cloud resources, simplifying multi-cloud infrastructure provisioning and management.
Multi-cloud and Hybrid Cloud Support: Crossplane supports multiple cloud providers (AWS, GCP, Azure, Alibaba Cloud, and more), enabling you to deploy workloads and services across different clouds, facilitating complex multi-cloud and hybrid cloud architectures.

Get Started

To provision an S3 bucket using Crossplane, we should have:

An existing Kubernetes cluster running (preferably minikube or kind)
Helm (to install Crossplane)
Configure the AWS provider

After following all the prerequisites, I have defined a custom resource definition (CRD) to provision an S3 bucket below:

After running [.code]kubectl apply[.code] command on our CRD, we see that our bucket is provisioned:

Licensing
Crossplane is open source and licensed under Apache License 2.0.

Ansible IaC Software for Configuration Management

Ansible is an open-source automation tool, or platform, used for IT tasks such as configuration management, application deployment, intra-service orchestration, and provisioning. Ansible uses a simple syntax written in YAML called playbooks.

Key features:

Agentless Architecture: No need to install any agents on the nodes to manage them.
Idempotency: Ensures that an operation will produce the same results if executed multiple times on the same system.
Simplifies Complex Deployments: Ansible's modular and scalable nature allows for the orchestration of complex multi-tier IT application environments.

Get Started

Taking an example to create an IAM using Ansible, you would typically use the [.code]aws.iam_user[.code] module. Here's a simple example of an Ansible playbook that creates an IAM user env zero-ansible-user:

Run [.code]ansible-playbook [playbook-name][.code], in our case iam-playbook:

Licensing
Ansible is open source and licensed under the GNU General Public License v3.0 (GPLv3).

Salt IaC Tools for Automation

SaltStack, also known simply as Salt, is an open-source configuration management and remote execution tool. It is designed to automate the management of infrastructure changes, software deployment, and configuration across a wide range of systems in data centers and cloud environments.

Key features:

Configuration Management: Employs a declarative, YAML-based language for idempotent state files, enabling simple, readable system state definitions with consistent outcomes on multiple applications.
Remote Execution: Efficiently executes commands across thousands of servers using ZeroMQ or SSH, with flexible targeting based on hostname, metadata, or custom grains.

Get Started

Install Saltstack, and make sure you have the latest AWS CLI installed for provisioning AWS resources. After that, configure your AWS credentials.

Define your S3 bucket config in the provision-s3.sls file in /srv/salt/ folder (make one, if it doesn’t exist) and make sure to uncomment the config below located in /etc/salt/master folder:

Here is the bucket config:

We provisioned the env zero-salt-bucket by running the [.code]salt-call[.code] command:

Licensing
Saltstack, or Salt, is open source and licensed under Apache License 2.0.

Chef IaC Tool for Infrastructure Automation

Chef is a powerful automation platform that transforms infrastructure into code, automating how infrastructure is configured, deployed, and managed across a network, regardless of size. It's especially popular in cloud and server management contexts, where managing large numbers of servers efficiently and predictably is critical.

Key features:

Infrastructure as Code (IaC): Chef uses Ruby to write infrastructure configuration, allowing for automation, versioning, and testing of your IaC.
Automated Configuration: Automates server configuration and maintenance for both small and large-scale environments, ensuring uniform system setup across any number of servers.

Get Started

Install Chef Workstation and generate a cookbook. After that, navigate to the cookbook directory and start writing your recipe.

Here is a recipe written in Ruby to provision an S3 bucket using Chef:

We successfully provisioned env zero-chef-bucket, running the [.code]chef-client[.code] command:

This is the Chef automated dashboard in action:

Licensing
Chef is available under Apache License 2.0.

Puppet IaC Tools for Infrastructure Management

Puppet is a configuration management tool designed to automate the management of infrastructure across its lifecycle, from provisioning and managing infrastructure configuration, to orchestration and reporting

Key features:

Automated Infrastructure Management: Uses a declarative language (Ruby) to automate infrastructure setup and enable centralized control.
Scalability: Employs a master-agent architecture, enabling it to manage a large number of nodes from a central point.
Security and Compliance: Automates compliance enforcement and security management by monitoring and adjusting configurations to meet external regulations, internal policies, and applying security updates.

Get Started

You can write a Puppet manifest to provision an S3 bucket like below:

We can visualize our infrastructure using Puppet Dashboard.

Licensing
Puppet is licensed under Apache License 2.0.

Alternative to Terraform: What Usually Triggers a Switch

Teams typically evaluate alternatives to Terraform when licensing, workflow requirements, governance needs, or multi-tool support become a priority.

Frequently Asked Questions/FAQs

Q. What is the difference between Pulumi, Crossplane, OpenTofu, and Terraform?

Terraform, OpenTofu, and Pulumi are Infrastructure as Code (IaC) tools designed to provision and manage cloud infrastructure, using HCL and popular programming languages (Python, JavaScript, .NET, etc.), respectively.

In contrast, Crossplane is built specifically for Kubernetes, allowing you to manage infrastructure through Kubernetes CRDs, encompassing a wide range of major cloud providers and their services.

Q. Which is better: Terraform or Ansible?

Terraform specializes in provisioning immutable cloud infrastructure with a declarative model, tracking all resources. Ansible is best for configuration management and application deployment with a procedural approach, without tracking resources. You should choose based on your focus: infra-provisioning or management.

Q. What is the difference between Chef, Puppet and Ansible?

Puppet and Chef are both configuration management tools that use a master-agent model and a declarative language for system configuration, requiring a setup process for the master and agents.

Ansible, on the other hand, is agentless, using SSH to connect and execute tasks in a procedural style, making it simpler to set up and use for ad-hoc task execution and automation.

Q. What tools are competitors of Terraform?

Different tools help us manage resources and automate infrastructure provisioning, like Terraform. Top competitors include OpenTofu (an open-source version of Terraform), Pulumi, and Crossplane, to name a few. Chef and Puppet, though competitors, are hard to install and configure and have more of a learning curve than Terraform.

Top Infrastructure as Code Tools and Terraform Alternatives

Igal Zeifman

env0 Marketing, Team OpenTofu

Blog

read time

min

The Ultimate Guide to Ansible Conditionals for Smarter Automation

Sam Gabrail

President, TeKanAid

Blog

read time

min

What is Ansible

Ansible is a powerful open-source automation platform that plays a key role in the world of configuration management, application deployment, and orchestration. It’s an essential tool in the platform engineering toolbox, enabling you to automate complex tasks with ease.

Ansible owes its popularity to its agentless architecture, human-readable YAML configuration files, and its ability to scale across thousands of nodes. These features make it a tool of choice to countless IT/DevOps organizations, looking for ways to efficiently automate their work processes and ensure consistency across their infrastructure.

In this tutorial, we'll cover the fundamentals of working with Ansible, from installation to creating your first playbook. In later sections, we'll dive into practical hands-on examples for some of the more advanced use cases and show why Ansible has become a go-to solution for many modern DevOps practices.

Video Guide

‍

TLDR: You can find the main repo here.

Why Use Ansible

Agentless architecture: Ansible operates without the need for agents on managed nodes, simplifying setup and minimizing maintenance efforts.
Human-readable YAML files: Ansible playbooks are written in YAML format, making them easy to read, write, and understand, even for beginners.
Wide platform support: Ansible works across various platforms, including Linux, Windows, cloud services, and networking devices.
Scalability: Ansible can manage thousands of nodes, from small setups to large-scale infrastructures.
Extensive Modules and Roles: Ansible offers a wide range of modules and roles for automating diverse tasks and easily reusing code.
Idempotency: Ansible ensures that applying the same configuration multiple times doesn't change the system's state after the first successful run. This characteristic is a must for maintaining consistency across environments. I will show you this in action in the demo.

Ansible Fundamentals

Now let's explore how Ansible works by reviewing its core concepts, before diving deeper into how they function:

Architecture

Control node: A system on which Ansible is installed. You run Ansible commands such as [.code]ansible-inventory[.code] on a control node.

Inventory: The inventory is created on the control node to define the hosts for Ansible to manage and deploy.

Managed node: A remote system, or host, that Ansible controls.

The Ansible Inventory File

An inventory is a list of hosts/nodes with IP addresses or hostnames.
The default location for inventory is /etc/ansible/hosts, but you can define a custom one in any directory.
You can configure inventory parameters per host, such as host, user, and SSH connection parameters.

Modules

Ansible copies and runs code or binaries on each managed node as needed, to perform tasks specified in your playbooks. Each module is designed for a specific function, such as managing users on a database or configuring VLAN interfaces on network devices. You can call a single module in a task or use multiple modules within a playbook. Ansible organizes these modules into collections, making it easier to manage and use them for various automation tasks.

Plugins

Ansible plugins are pieces of code that expand the core functionality of Ansible. There are plenty of handy plugins, and you can write your own plugins as well. You can find more details about Ansible plugins here.

Playbooks

Playbooks are the core execution units in Ansible, consisting of "Plays" that define which managed nodes (hosts) execute specific tasks.
Ansible Playbooks are a way of sending commands to remote systems through scripts.
Ansible playbooks configure complex system environments, enhancing flexibility by allowing scripts to be executed across multiple systems.
Playbooks are written in YAML format and outline the tasks to be performed by Ansible.

Plays

Plays map managed nodes to tasks, containing variables, roles, and a sequence of tasks. They define how to iterate over these tasks for each host.

Roles

Roles bundle reusable content like tasks, handlers, and variables for use within a Play.

Tasks

Tasks specify actions to execute on managed hosts. They can be run individually using ad hoc commands.

Handlers

Handlers are special tasks triggered only when notified by a previous task that has made a change.

Working with Ansible Roles

Ansible roles are a powerful way to organize and manage your playbooks, making your automation more modular, reusable, and maintainable.

Using roles, you can group tasks, variables, files, and handlers into separate, self-contained units. This allows you to share functionality across different teams, environments, or projects, helping to keep your Infrastructure as Code (IaC) clean, organized, and DRY (Don't Repeat Yourself).

Roles simplify playbook management and take automation to the next level of abstraction. Instead of cluttering a single playbook with all the details, you can break down your tasks into roles and then call these roles from your playbook, ensuring a streamlined and scalable approach to automation.

While working with Ansible roles can greatly enhance your automation efforts, it is beyond the scope of this particular blog post. I will explore Ansible roles in more detail in a future article.

Managing Your Ansible Server

The machine where Ansible is installed and from which all tasks and playbooks are run is sometimes called the Ansible server or the Ansible Control Node.
Ansible executes modules on the managed nodes, which are invoked by tasks.

This server would typically be a runner in your CI/CD pipeline.

How to Install Ansible

Ansible can be installed on various operating systems. You can install Ansible using the command line with the package manager of your operating system. You can find more details in the official installation guide.
To keep things simple, as you follow along with this Ansible tutorial, you can easily start a GitHub Codespace with Ansible installed from this article's repo.

Ansible Tutorial Demo

In this demo tutorial, we'll explore how to automate the setup of a Jenkins CI/CD environment on an EC2 instance using Terraform for provisioning and Ansible for configuration. Additionally, we'll leverage Docker to encapsulate Jenkins in a container, ensuring a portable and isolated environment.

Below is a diagram of what you will build:

Tools Overview

Before diving into the tutorial, let's briefly overview the tools we will be using:

Terraform: You'll use Terraform to automate the creation of your EC2 instance and related networking resources. We won't spend too much time explaining Terraform since our focus is on Ansible.
Ansible: Ansible will install Docker, configure the Jenkins environment, and run Jenkins inside a Docker container on the EC2 instance.
Docker: You will use Docker to run Jenkins as a container on your EC2 instance.

Step 1: Provision Infrastructure with Terraform

The first step is to create the infrastructure needed for our Jenkins environment using Terraform.

Navigate to the Terraform directory: Start by navigating to the directory where your Terraform configuration files are located.

Initialize Terraform: Initialize your Terraform environment to download the necessary provider plugins and prepare your working directory.

Apply the Terraform configuration: Apply the Terraform configuration to create your AWS infrastructure. The [.code]-auto-approve[.code] flag will bypass manual approval for the changes. Be careful using this flag in production environments.

Terraform will create the following resources:

A VPC (Virtual Private Cloud) with subnets
Security groups to control access to your EC2 instance
An EC2 instance where Jenkins will be installed
An Elastic IP for public access to the instance

After the process is completed, Terraform will output critical information, such as the public IP address of the EC2 instance and a private SSH key for secure access.

Step 2: Prepare Ansible Inventory and SSH Key

With the EC2 instance created, we need to prepare the Ansible inventory file, which Ansible uses to know which hosts to manage.

Additionally, we’ll prepare the SSH key that Ansible will use to authenticate with the EC2 instance.

Update the Ansible Inventory file: Replace the placeholder in the Ansible inventory file with the actual public IP address of your EC2 instance. This can be done using a simple [.code]sed[.code] command.

After updating, verify the contents of the inventory file to ensure that the IP address has been correctly inserted.

Extract and prepare the SSH key: Extract the private key generated by Terraform and save it to a file that Ansible can use to connect to the EC2 instance.

Set the correct permissions on the private key file to secure it.

Step 3: Configure the EC2 instance with Ansible

Now that the infrastructure is in place and the inventory and SSH key are prepared, you can move on to configuring the EC2 instance using Ansible.

Navigate to the Ansible directory: Change your directory to where the Ansible playbook is located.

Run the Ansible playbook: Execute the Ansible playbook to configure the EC2 instance. This playbook will install Docker, set up the Jenkins container, and ensure that everything is running correctly.

The playbook playbook.yaml file consists of several tasks that automate the following steps, more details will be provided in a later section:

Install pip3 and unzip: These are required for installing Python packages and handling compressed files.
Add Docker GPG apt Key: Adds the GPG key for the official Docker repository.
Add Docker Repository: Configures the Docker repository in your package manager.
Install Docker: Installs the latest version of Docker CE (Community Edition).
Install Docker Module for Python: Installs the Docker module for Python, enabling Ansible to manage Docker containers.
Pull Jenkins Docker Image: Pulls a pre-built Jenkins Docker image from Docker Hub.
Set Permissions: Ensures the correct ownership and permissions for Jenkins data directories.
Create and Start Jenkins Container: Creates and starts the Jenkins container, mapping necessary ports and volumes.

Step 4: Verify Jenkins Setup

Once the Ansible playbook has completed its execution, Jenkins should be up and running on your EC2 instance. Let’s verify that everything is set up correctly.

Access Jenkins: Open a web browser and navigate to the public IP address of your EC2 instance using port 8080.

This will bring up the Jenkins login page:

Retrieve Jenkins Admin password: The initial Jenkins admin password is stored on the EC2 instance. SSH into the instance to retrieve it as shown below, replacing ‘public_ip’ with your public IP.

Get the password:

Copy this password and use it to log into Jenkins.

Complete Jenkins setup: After logging in, follow the Jenkins setup wizard to install recommended plugins or skip the plugin installation to expedite the process. Once completed, your Jenkins instance will be ready to use.

Inventory File and the Ansible Playbook: A Deeper Dive

The Inventory File

To set up your Ansible environment, you need to create an inventory file that stores client details, such as hostnames or IP addresses and SSH ports.
You can use SSH keys to connect to clients and simplify the process.

Ansible uses the inventory file to manage remote machines and network devices.

Here is the inventory file that you used in the demo with comments:

Playbooks vs. Ad Hoc Commands

You can use Ansible ad hoc commands to issue some commands on one or several servers. They are useful for tasks you rarely repeat. Therefore, you will mostly resort to using Playbooks.

To write your playbook, you need to define the desired state of a system using Ansible playbook commands. Below is the playbook.yaml file with comments.

Integrating Ansible with env0

Integrating Ansible with env zero offers an efficient way to automate infrastructure management, leveraging the power of templates to streamline the process.

Instead of running commands manually through the CLI, you can create and manage environments directly within env0, simplifying the entire deployment process.

To demonstrate this, let's walk through how you can use env zero to replicate the same setup we previously configured using the CLI. The first step is to create a project within env0. Let's call it “Ansible Tutorial”.

You can define a new environment within this project based on a custom Ansible template. This template includes the necessary configurations, such as the Ansible version (which can be set to a specific version or left as the latest) and the SSH key.

The SSH key used here corresponds to the private key generated earlier with Terraform, ensuring secure access to your EC2 instance.

The next step involves linking this environment to your GitHub repository, specifying the folder where your Ansible playbook resides. This ensures that env zero knows where to find the necessary scripts and files to execute the automation.

You’ll also need to define environment variables, such as ‘ANSIBLE_CLI_inventory’, which tells Ansible the name of the inventory file to use. With these settings in place, you're ready to create and run your environment.

Once you initiate the run, env zero takes over, cloning the repository, setting up the working directory, loading variables, and executing the playbook. Since the environment setup was already handled via the CLI, env zero will verify that all configurations are correct.

After approving the run, env zero executes the playbook, mirroring the steps we performed earlier. The process concludes with a successful deployment, and you can review the logs to confirm that everything has been configured correctly.

The above example showcases how env zero enhances the Ansible experience, automating repetitive tasks and providing additional benefits such as version control, collaboration features, a host of integration options, and governance features like RBAC and OPA policies.

These features make env zero an excellent tool for managing complex infrastructure deployments using any popular IaC framework or a combination of frameworks such as Terraform, OpenTofu, Pulumi, CloudFormation, and more.

To learn more about how env0’s platform can help with governance and automation check out this guide: The Four Stages of Infrastructure as Code (IaC) Automation.

Conclusion

This guide highlighted the power of the Ansible Automation Platform in configuration management and application deployment through its agentless architecture and straightforward YAML playbooks.

In this blog, we focused on setting up a Jenkins CI/CD environment on AWS, where Ansible automated tasks like installing Docker and managing containers, ensuring consistent and efficient infrastructure setup.

While Terraform was briefly used to provision the AWS resources, Ansible played the central role in configuring the environment, showcasing its ability to streamline complex tasks with minimal manual input.

Finally, we also touched on integrating Ansible with env0, which further enhances automation by providing version control and environment management tools. Together, Ansible and Terraform offer a comprehensive solution for efficient infrastructure automation.

Frequently Asked Questions

Q: What is the difference between Ansible and Jenkins?‍

Ansible is an automation tool primarily for configuration management, whereas Jenkins is a CI/CD tool focused on automating software builds, tests, and deployments. They serve different purposes but can complement each other. Here’s a quick comparison:

	Ansible	Jenkins
Use Case	Infrastructure setup, app deployment	Building, testing, deploying software
Setup	Agentless	Requires agents on nodes
Language	YAML	Groovy, Shell Scripts

Q: How do I create a directory in Ansible?

To create a directory in Ansible, use the ‘file’ module with [.code]state: directory[.code]. Example:

Q: Can Ansible be used with Windows?‍

Yes, Ansible can manage Windows servers. You can use modules like ‘win_shell’, and others specifically designed for Windows. You need to set up WinRM on your Windows machines for Ansible to communicate with them. Check out the docs: Using Ansible and Windows

Q: How does Ansible compare to Terraform?

‍Ansible and Terraform are better together. Terraform is ideal for provisioning infrastructure, while Ansible excels at configuring that infrastructure once it's up. Both tools allow you to automate your entire environment from start to finish. You can learn more here: Ansible vs Terraform: Choose One or Use Both?

Q: How do I start learning Ansible?‍

Start by learning the basics of YAML format and command-line operations. Explore Ansible’s documentation and try hands-on labs to create simple playbooks and roles. This blog post is a great starting point!

Q: Is Ansible easy to learn?‍

Yes, Ansible is considered easy to learn, especially for those familiar with basic system administration. Its use of YAML format for playbooks makes it accessible for beginners.

Q: How long does it take to learn Ansible?‍

It depends on your background, but you can learn the basics in a few days. Mastering Ansible for complex environments may take a few weeks to a few months, depending on the depth of your learning and practice.

Q: Should I learn Python or Ansible?‍

It depends on your goals. If you want to automate infrastructure tasks, Ansible is the way to go. If you're interested in more general-purpose programming and scripting, Python is essential. Knowing both can be extremely powerful as Ansible is written in Python, and you can extend Ansible with Python scripts.

The Essential Ansible Tutorial: A Step by Step Guide

Sam Gabrail

President, TeKanAid

Webinars

read time

min

A new year and tricky economic times seemed like the perfect opportunity to gather some IaC experts for a conversation about hiring challenges, where we’re headed, and doing more with less.

In case you missed our infrastructure as code webinar, here’s a bit of what I learned in my conversation with Brandt Meyers, enterprise architect with MGM Resorts International, Kat Cosgrove, lead developer advocate at Dell, and Chris Short, senior developer advocate at AWS.

Code once, reuse often

Market research firm Gartner Group says infrastructure as code will be the most in demand skill this year, so how can teams handle this? One great option, suggested by Brandt, is to change the corporate mindset, moving from prescriptive code creation for a single use to subscriptive coding where code is created to be reused within an organization and perhaps beyond. While hiring and retaining still remain challenging, changing the code creation strategy could make a difference.

The continuing power of automation

Also making a huge difference, particularly to job seekers in this uncertain market, is a deep understanding of automation. “There is always a need for people who know how to automate,” Chris said. “If you’re good at automating you’re going to have a job.” Chris said he thinks a lot of small startups are continuing to hire and that they’re looking for people with infrastructure as code or platform engineering expertise. I agree with that, but in my experience those skills continue to be difficult to find.

Start with infrastructure as code

It’s Kat’s hope for 2023 that teams will finally think infrastructure as code *before* they even start to build a proof of concept. “At this point in the industry IaC is no longer an afterthought,” she said. “It’s time for everyone to accept IaC as a requirement when building an app.” Those who ignore that advice will find “bolted-on-later” IaC difficult to do, she warned.

Want collaboration? Try Everything as Code

One of Brandt’s team goals for the year is increased collaboration and his approach is novel: put code at the center. “It really goes back to the fact that IaC is evolving into EaC (Everything as Code),” he explained. “You want to maintain the configuration and make everything consistent. Code is a way for us to collaborate and have a common language.”

How to do more with less? Cross-train

We all agreed that, thanks to the uncertain economy, many many teams will be trying to figure out how to be more productive with fewer resources. And a good place to leverage that is with infrastructure as code, Chris said. “We need the consistency that IaC brings. The more people who adopt it the better,” he said. To ensure all team members can appreciate the benefits, Chris suggested cross-training. “Learn a bit about infrastructure if you’re a dev, and if you’re on the infrastructure side get familiar with GitHub or Python. The more we have crossover like that the better things will get. We need to share these best practices more.”

Stick to what you do well

And finally, it’s always important to remember that there is not one magic tool…for anything. “I want to see more companies not trying to be like a pocket multi-tool, because there isn’t one tool that is the best for all these specific things,” Kat said. Her take, and I agree with it: Specialize in what you’re good at and actively collaborate with other tools that fill in the gaps, or in other words, “stop trying to force it.”

Watch the full 2023 Infrastructure as Code roundtable

Webinar Transcription:

Justin Nemmers: 00:02:28

‍All right, hello. Welcome. Good morning, good evening. To everyone today, I'm really excited to be bringing a fantastic panel of, uh, folks here to talk about infrastructure as code 2023. You know, what are we seeing? What lies ahead? Uh, and like, why is it a big deal? So, uh, before we get into the actual webinar itself and the panel discussion, I want to go over a little bit of housekeeping. Of course, all of your microphones and cameras have been muted. Um, it's not that we don't wanna see or hear you, but, uh, you know, it's just kind of chaotic, uh, moving forward. If you have a question, please use the q and a button on your, uh, zoom webinar panel there, and you should be able to post a question. Um, some of those will be able to get in line. Most of them we will, uh, likely end up answering at the end of the actual discussion. And so with that, uh, by all means, let's get into the main program. So for starters, introductions, who are we even talking about here? Who, who's on the panel? Uh, who are they and why they important. So let's get that part underway. So Brandt, why don't you, uh, tell everyone a little bit about yourself?

Brandt Meyers: 00:03:32

Sure. Yeah. So I'm with MGM Resorts International and Enterprise Architecture Group. I'm responsible for our cloud reference architecture, our DevOps practice and automation. And I'm working on our, in my journey. I'm working on my fifth generation of infrastructure code with MGM.

Justin Nemmers: 00:03:55

‍Fantastic. Thank you. Uh, Chris,

Chris Short: 00:03:58

‍Everyone. I'm Chris Short. Uh, I'm a senior developer advocate at AWS on our Kubernetes service EKS, um, because our acronyms are easy to remember. Um, the <laugh> I've spent the past 23 years now in tech. It feels like something like that longer probably. But, uh, a big background in DevOps, and I'm a Kubernetes contributor as well.

Justin Nemmers: 00:04:24

Awesome. Awesome. Kat?

Kat Cosgrove: 00:04:27

‍Hello. Hello. Uh, my name is Kat Cosgrove. I'm a lead developer advocate at Dell in our brand spanking a new, uh, super fancy DevOps team. Um, I have been around for a while, but mostly, um, in not DevRel. DevRel thing is, uh, relatively new to me over the last, they've been doing that for like four years, but I used to be an engineer. Um, I live in Seattle with my two cats, one of whom is with me on this webinar. Her name is Espresso, um, <laugh>, and I am also a Kubernetes contributor alongside Chris.

Justin Nemmers: 00:05:05

‍Awesome. Thank you. Uh, and awesome guest appearance by, uh, Espresso there. Ohad.

Ohad Maislish: 00:05:12

‍Hi everybody. Ohad Maislish, co-founder and CEO at env0. If you've not heard about env zero yet, we care about infrastructure as code is this, uh, this panel we provide management solution on top of the frameworks on top of Terraform, Pulumi, CloudFormation, and, uh, and others. I'm originally an engineer, uh, moved to, uh, be founder and I like to talk about tech. I'm still always was Geek <laugh>.

Justin Nemmers: 00:05:41

‍I love it. And, uh, the interesting part about this is we are all either current or recovering engineers on this, uh, on this panel. So, uh, very true. It's fantastic recovering, always recovering <laugh>. Awesome. Um, alright, well thank you so much for joining us. So let's just go ahead and dive into it. We've got a, uh, I think a pretty good set of discussion topics here that we're gonna dive into. And, and I'll set the stage a little bit at first and then we will have, um, have some around the horn here. Uh, so for starters, as server automation adoption accelerated largely thanks to tools like Ansible, the workplace demand for automation talent significantly outpaced the supply. And now for the first time ever, we're beginning to see that same trend happen with infrastructure as code. So Gartner is predicting that IaC will be the most in demand skill in 2023. And the question ultimately is how can you prepare for this? Uh, we all have teams that we need to do and work with. Um, and when that skills gap is present, it, it makes things especially challenging. So, to get started here, why don't I pass it over to you, Brandt, for some of your thoughts.

Brandt Meyers: 00:06:45

‍Yeah, so I think, um, you know, a couple opportunities there is, is working with our existing engineers and, and giving 'em opportunity to learn and, um, you know, really supporting that, um, bringing in talent is, is a challenge in keeping talent. I think all companies are working on that. Um, and um, I think another piece too is with an infrastructure as code changing a mindset from prescriptive to subs descriptive where, um, when we build code that's prescriptive, it's, um, it's for a particular use case and it's not really reusable, but, um, building code that's more extensible and that can be, uh, reusable across an organization first. Um, instead of teams building specifically for their use case, they build for the organization where you can reuse with an organization and then maturing to a point as an industry where we can reuse and, and leverage, um, our talent collectively.

Justin Nemmers: 00:07:53

‍Fantastic. Now, you know, Chris, I'd imagine in your role you end up seeing a pretty broad, uh, number of just customers and organizations that, uh, that in some cases are, are, are trimming some folks, and we're, we're gonna talk about that momentarily, but it seems like for every one company that you're hearing about layoffs from, there are five more that are, that are doing a, a wild amount of hiring. I mean, so how do you, how do you think the dynamic that brand is talking about really will continue to play out?

Chris Short: 00:08:21

‍I mean, there's always gonna be a need for people that can automate things, right? Like, if you're good at automating things, you have a job, right? <laugh> just, just understanding what it takes to automate a business logic in applications or infrastructure is gonna take you a long way. But yes, there is this odd economic situation that we're kind of toiling with right now, uh, especially in large tech companies. Um, but it seems all the small startups that I've talked to are still hiring, which is interesting. And those startups are looking people with infrastructures as code skills or, you know, platform engineering type skills, I think is what it's being referred to a lot where there's a developer platform that a team maintains and, you know, that can be a team of SREs or whatever. Um, those skills are gonna continue to be in demand. And what we're gonna start seeing now, especially with like the downturn in ed revenue, is that information is gonna be like a key thing, um, that you're going to have to maybe struggle to find. Or in the past it was kind of pushed into your face. So being able to discover trends and new tooling or new best practice, not best practices, but new practices that, uh, improve on your existing ones will take you a long way no matter where you go. Now, companies finding those people, that's hard because normally, I mean, I, I speak, I'm speaking for myself here, after about 10 years in tech, you're referring to your network mostly, um, to get jobs and not necessarily like trolling the internet for 'em or job boards especially. So yeah, it's, it's hard to get influence with these people to a say, use our tool, let alone get them to actually come on board, right? Like, that's a whole nother challenge in and of itself. And I'm sure Ohad has all kinds of stories about that.

Justin Nemmers: 00:10:24

‍Yeah. Well, I mean, great pitch over. So Ohad, you know, what, what are some of the, the things that you've seen regarding this? I mean, we, we've clearly, everyone has a need for personnel on some level, uh, and when the, the skills don't meet, meet the, the actual requirements, what, what do you do?

Ohad Maislish: 00:10:40

‍You have, you have a problem? I, I remember, uh, one thing I'll never forget when, Omry and I started env0, we started, uh, to do some market research and we, uh, scheduled a meeting with a friend of mine who is a VP engineering in, uh, in a big company managing, uh, about 100 engineers. And the meeting was for like one hour and he just, uh, allocated more time to talk with us. It was like close to two hours. He had a big office, uh, big shot. Uh, and then when he started asking more and more questions, he said, I, I don't know those answer. He's or, uh, Head of DevOps, Head of SRE, he can answer those questions. And then I will never forget that we ask him, can you bring him to the room for like five minutes so we can ask him? He said, oh, no, no, he's, he's busy. Don't interfere with, uh, with his time. And I'm like looking that you're managing like 100 people and eventually the bottleneck is your, uh, is your DevOps team. So there's definitely a clear <laugh> a clear problem here that those skills are both needed and difficult to, to find the relevant people that can actually do this job. The need for automation constantly increases. Chef, Puppet, and as you mentioned, Ansible was one generation of configuration management. Now we see, uh, similar and probably a bigger thing with infrastructure as code. And the problem is still very much existing out there, in my opinion.

Chris Short: 00:12:12

‍And Kat, I think you can talk to how the <laugh> the, the landscape has changed over those years, right?

Kat Cosgrove: 00:12:19

‍Yeah. So when things like Ansible and Chef first came out, you know, um, we, we called that configuration management and it was, um, configuration management at the time. The, the term infrastructure as code didn't really exist, even though we've had things that do kind of that for, um, literally decades. We've, we've had something like automated configuration management as long as we've had computers longer than we've had computers. Um, where things changed is when we started, um, lumping more pieces of automation in with just managing configuration. And now we have a term for that infrastructure as code, but, um, configuration management tools like Ansible, although they predate infrastructure as code as a term, they are infrastructure as code tools now, right? Like we consider them that now we didn't used to, but, um, configuration management is now just like a, a subset of infrastructure as as code. Like it's just become an umbrella term for you're automating anything to do with, uh, the infrastructure that an application is running on, whether that is standing up the entirety of the infrastructure or just configuring additional infrastructure that was stood up with something else. Um, which is like the, the issue with hiring, um, and the, the, the lack of people that have these skills, even though we increasingly need these skills, I think is why we're seeing more products like Pulumi and CDK, um, rising to popularity. It allows you to get your engineering team more easily onboarded onto the concept of infrastructure as code than if they had to learn like Terraform or something. If they're not used to slinging YAML or like writing HCL, then maybe writing Python is a little bit easier. Like, I know Chris came from a more ops background than, than I did. So I'm way more comfortable with like, Pulumi, CDK, it was way easier for me to transition into doing infrastructure as code with those tools than it was with Terraform. But I bet Chris had the opposite experience.

Chris Short: 00:14:40

‍I mean, I remember reading the original YAML spec cuz we were switching over to Puppet, right? And then, uh, quickly discovering how awful Puppet was, um, at the time. At the time, yeah. It required so much resources, you know, back now it's like, oh, it's just Puppet, it just, it has a box by itself somewhere kind of deal. And, but back then it required a ton of infrastructure. So, you know, managing that aspect alone was hard. But yeah, slinging and YAML was what, you know, my bread and butter was for many, many years where Yeah.

Chris Short: 00:15:14

‍Was more slinging structures and functions. Right. <laugh>.

Justin Nemmers: 00:15:18

‍So I'm kinda curious, you know, Brandt, how does this play out on your team? So, you know, you've, you've indicated that you've dealt with, uh, four now five completely different generations of, uh, of Terraform implementation. You know, presumably you didn't magically just grab a team that suddenly was all known around Terraform in this case, although as great as that would be, like, you know, you just press a button or, you know, run a YAML file and, and magically new new resources appear <laugh> in the form of humans to jam out more. Terraform clearly doesn't work like that. So how, how have you seen this practically play out?

Brandt Meyers: 00:15:54

‍So yeah, it's kind of, um, working out a facilitating relationship for the need and, and, um, one of the, one of the things we've done is we've partnered with, um, like Ohad with env zero right? To, um, to help facilitate those, uh, fundamental capabilities that we need to be able to, um, to drive the solution. And so having that foundation of, um, you know, those, those value adds that, um, you can get right out of the gate is really important. But also being able to have, um, collaboration, um, there, there's, there's, there's four Cs that I've, are, are in my mind at this point that are, um, are focused for 2023. There's, there's cost collaboration, there's um, there's the consistency and um, there's, there's the, uh, I'm trying to remember what the fourth one is now. There's, there's also the, um, kind of the, there's continual improvement. Um, but just, just kind of having those elements in mind this year is really important. And we kind of get that with having a facilitating partnership with, with env0, but also within our company, creating a DevOps organization and facilitating across teams to build a common practice.

Justin Nemmers: 00:17:27

Brilliant. Yeah, I mean, as much as we would always like to say that there's gonna be one magic tool that's going to solve everything for everyone, the reality is, is definitely not the case.

Kat Cosgrove: 00:17:36

‍It doesn't exist,

Kat Cosgrove: 00:17:38

‍Vendor, it doesn't exist in any category of technology. Like full stop, there is no one best cloud provider. There is no one best programming language. There is not and never will be one best IaC tool.

Justin Nemmers: 00:17:49

‍Oh yeah, there was a world's best programming language and it's called Perl. Um, alright, thank you very much everyone. This is. No totally kidding.

Chris Short: 00:17:57

‍Please. Oh my gosh. As someone that maintain a CMS completely written in Pearl for a newspaper company, I disagree. Yeah,

Justin Nemmers: 00:18:04

‍That's right. Um, alright, so like, all kidding aside, let's, uh, let's just kind of dig into this a little bit. So, you know, we, we've talked a bit about like, how do we handle this, like glut of resource, or not glu, but a a really a glut of need. Everyone has need for more talent on this front. Uh, so that's a trend clearly that's going to predictably continue. I mean, what else? Like, what else do you, do you think that's gonna become more prominent around IaC uh, over time and like for instance, uh, and we might actually just end up asking this as a poll, I think, but, but you know, we've seen over the past couple of years, Kat, you did a great buildup to this talking about how Ansible originally was being used basically as a, an IaC tool. Yeah. Uh, although there was no such thing as IaC initially. Over time we've begun, begun to see many, many more options. You know, in the end, in the beginning it was, it was pretty much just Ansible, you know, maybe Chef and Puppet, uh, then, um, Terraform, and now they're kind of a bunch of choices. So we're actually kind of curious as to what, uh, the audience here is running. So let's go ahead and publish that poll and then we will kind of continue to, to talk through this as, uh, as, as votes come in. Yeah. So what are some of the trends that we're gonna see kind of continue to, uh, to increase here? Ohad, what do you think?

Ohad Maislish: 00:19:20

‍No, I just wanted to comment that if I remember correctly, Puppet was recently acquired and if, if I remember correctly, I think the headline in TechCrunch or another place was infrastructure company Puppet got acquired by such and such. So that's, uh, an interest thing how to, to look at the old configuration management frameworks as, as Kat mentioned earlier, of, uh, of the first version of infrastructure as code in a way, eh,

Kat Cosgrove: 00:19:45

‍Puppet did get acquired, but I don't remember by whom.

Chris Short: 00:19:50

‍Yeah, it was like an investment firm

Kat Cosgrove: 00:19:51

‍Or something, wasn't it? And also, uh, Ansible wasn't the first. Um, like you ages before we had the Lake, Ansible, Chef, Puppet wars, um, everybody used CF engine, um, which still exists.

Justin Nemmers: 00:20:06

‍Yes, it does.

Kat Cosgrove: 00:20:07

‍Uh, crazy. Yeah, still exists, still exists really hard to use. Learning curve is outrageous. Um, but at the time it was absolutely revolutionary.

Justin Nemmers: 00:20:21

Oh, I, I remember it well, <laugh> user, and they, uh, like we'd go into an account when I was back with Ansible and like, oh, what are you using? And yeah, we heard that one a lot. Uh, follow by a groan because yeah. Uh, if you don't like writing HCL, imagine writing C to define your configurations. Definitely a blast.

Kat Cosgrove: 00:20:41

‍Yeah. I never had the pleasure, fortunately,

Justin Nemmers: 00:20:46

‍Yeah. So, you know, but on that note, Ohad, uh, to kind of continue on. So, so what, what are some of the things that you think are gonna continue to be prominent within the IaC community in 2023?

Ohad Maislish: 00:20:57

‍I think Terraform is, uh, having a lot of, a lot of success. Um, when we started env0, we didn't know where the focus, when the focus will with Terraform customers or maybe Pulumi later. We haven’t heard more about Crossplane coming up in this conversation today. Uh, but I can, my, my feeling is that Terraform continues to, to be that defacto leader in, uh, in the IaC world. Uh, although the, you know, the, the issues we talked earlier about how to open, uh, you know, the ability to write infrastructure as code to wider audience of engineers, uh, the fact that he has has such a wider, wide ecosystem with, uh, so many providers and models and community, uh, is really helpful to look at that as an end-to-end solution. And when we even talk with single cloud customers like AWS customers or Azure customers, I think like three, four years ago we heard more CloudFormation, we heard more ARM templates. Uh, you know, I, I wanna share that I think like two years ago I had the honor to talk with CTO of Azure, uh, about ARM versus Terraform. And ah, well Terraform went doing very well on the poll. Uh, and he was very much focused on, on ARM. And he said that Azure customers will just use ARM and they don't need Terraform. And I think like three, four months ago, um, they released a very interesting open source like Terraformer, but specifically for Azure, which basically takes all of your, uh, Azure Cloud resources and automatically generates Terraform code. So even Azure team, uh, has realized that their customers pay a lot of attention to Terraform, not necessarily for ARM, and their HCL based version named Bicep. Not sure if people here know about Bicep, it's a more HCL version of, of ARM, but it's not Terraform. So eventually even Azure customers probably use more and more Terraform. We see that with CloudFormation being less used, more, more Terraform. So I think what I'm trying to say here, I've mentioned the word Terrafom like 10 times in the last three minutes. So that's, I think is a multicloud open source with a huge, uh, ecosystem of providers, public providers, and public models. I think it's, uh, it's the clear trend that we continue to see over and over.

Justin Nemmers: 00:23:29

‍Yeah. And you know, it's interesting. So the one thing I, I was wondering if you could possibly say a couple of words about Terragrunt. So I know that we see a fair amount of that at env zero as well. Uh, and clearly it's, it's pretty prominent in their response here. I think most people probably know what CloudFormation is Terraform, uh, but Terragrunt might be the outlier there where, you know, there's a total percentage of usage out there. It's pretty small, but we see, I think a lot of it growing. So, Ohad. What, what are your thoughts on that?

Ohad Maislish: 00:23:54

‍Terragrunt is always there. I don't remember how old Terragrunt is, but, uh, it's a more advanced, uh, flavor. Sometimes I say it's the cousin of, of, of Terraform in a way, uh, do not repeat yourself kind of framework. And we see a lot of, DevOps engineers that try to understand how to map everything to infrastructure as code when they have the nuance of, uh, choosing either Terraform or Terragrunt, very often trend towards, uh, Terragrunt I do have a feeling that it becomes less crucial as it used to be a few years ago, because the Terraform framework has evolved with some of the key things that Terragrunt and had while Terraform still didn't have. But still, we very often see, uh, Terragrunt users, uh, having some additional capabilities that Terraform is, uh, is lacking. And I think in general, Gruntwork is doing a, a very, very good job, both the Terragrunt and their, uh, the other solutions of Terraform models that they, that they provide. And maybe that's yet another reason why Terragrunt is doing successfully, uh, thanks to the great local, uh, Gruntwork.

Justin Nemmers: 00:25:09

‍Excellent. Kat, what do you think is gonna increase in 2023?

Kat Cosgrove: 00:25:13

‍Uh, um, I think hopefully we'll see more people thinking about infrastructure as code from the outset rather than trying to shoehorn it in later on. Um, that, that is hard to do and it introduces more problems than are, are necessary, I think. Um, so, so hopefully we see people just like day one consideration is we are going to use an infrastructure as code approach to build and deploy this application. Um, rather than building a proof of concept on manually configured, uh, infrastructure and then trying to switch over later on, uh, in a scramble to find people to hire to do it for them. Um, it's just, it kind of feels at this point, um, in the industry, like infrastructure as code is no longer an afterthought. It is just a given. That is the way we operate now. Um, standing up and configuring infrastructure by hand is, uh, inefficient, slow, and dangerous, and everybody kind of just like seems to accept that now. Um, so hopefully this is the, the beginning of us just accepting it as a default, just a requirement of building an application.

Justin Nemmers: 00:26:31

‍Yeah. So Brandt, it seems like you typically agree with that, given the head nods, but, you know, I guess I'm just kind of curious for 2023, you talked about your four Cs, you know, is there any one of those between, uh, let's say I wrote 'em down, cost consistency and continual improvement? Um, as a marketing guy, I definitely, um, re recovering engineer, but marketing guy, I definitely love the, uh, <laugh> nice bite size components there. Which of those do you, do you think is gonna be most important for your team? And, and how does IaC play into it?

Brandt Meyers: 00:27:01

‍I think collaboration is key. And, and really it is, it goes back to like, I think it's just IaC is gonna evolve into EaC. Everything is code. Um, and, and, and it's not just, and it doesn't stop at cloud. It's, it's everything. It's even SaaS. Um, it's, it's maintaining a configuration of everything consistently and, and being able to bring back together, uh, a technology organization. Um, what I've seen is, and, and I've experienced this, you know, a number of times where it's just, um, we, we have this radical shift and some teams focused on that shift, and other teams are focused on sustainment or iteration, not necessarily innovation. And so, you know, I think, I think organizations this year are gonna come back to how do we come together again, as, as a, as an organization and, and code, you know, as a way for us to collaborate and have a common language. And we need to decide what that language is and, and how we use that effectively.

Ohad Maislish: 00:28:06

‍I have to say, Brandt, um, I'm a close friend of, uh, founders of the company named Salto, and not sure if you've heard about Salto. Uh, they're backed by, uh, Accel and some other great, great investors. And they, the first time I've heard the term Company is Code, you mentioned everything is code, and they say company is code. And, uh, you've mentioned the, the increase of, uh, providers and SaaS and not just the actual cloud vendors. We see that with the Okta, New Relic, and Datadog. Even env zero has its own Terraform provider. But what Salto is trying to push towards is becoming fully managing everything with code, including things like Salesforce and Zendesk and your, uh, financial, uh, internal tools. Uh, that's really interesting to see how you no longer, uh, click a button to change some configuration of your business, but instead you write code. So that's, I think, uh, a trend that continues to happen.

Chris Short: 00:29:09

‍I think Linux just opened up that realm of doing things to everyone finally, right? We had years of Windows dominance, and then finally you could, like, the common person could build a Unix like system and touch config files on a regular basis. And that made more sense to them than clicking through a bunch of check boxes. And that evolution has continued on since the nineties, it feels like.

Justin Nemmers: 00:29:35

Yeah. And I think, you know, ultimately it's a, uh, it's a great one. So, you know, for every point there's a counterpoint, and I guess I'm, I'm kind of curious as what you all think, what's gonna go away, what are we gonna see less of, if anything in, in 2023? And you should know that, um, the, the, uh, opposite of the answer you already provided will not be acceptable <laugh>. Um, so yeah, I mean, what, Kat what do you think in 2023, what are we gonna see less of? What is gonna be less prominent or maybe you hourly disappear.

Kat Cosgrove: 00:30:07

‍There's, there's what I want to see less of and what I think we'll see less of, um, what I, what I want to see less of is I, I want to see less of, uh, vendor lock in. I, I hate it. I'm allergic to it. Um, I don't like using tools that force me to use a specific vendor. Um, which is, which is why I will prefer things like Terraform, Pulumi over a cloud provider's, like specific tooling. Um, I don't like that. I think it is, uh, unsafe. Um, that's not necessarily to say that I think that multi-cloud or hybrid cloud is the right answer for everybody, because that does introduce a ton of extra work also. But, um, I would like to see people more carefully choosing their tooling to leave the option open to not be locked in to something forever. Um, cause it, it is just, I don't know,

Chris Short: 00:31:11

‍I would even expand on that, Kat. You know, I feel like some engineers are gonna have tools of choices. Right. For sure. And I feel like we'll see maybe a stacking up of tools that become your holistic IaC thing. Someone mentioned in the q&a, IaC, Terraform and Ansible working together from time to time. I feel like that's gonna happen with the Pulumis and all the other things of the universe, Terraform, um, kind of in mass. Like people will have their tool and just as long as they can automate the tool doesn't matter as much.

Kat Cosgrove: 00:31:47

‍Yeah. And honestly, the thing that like, kind of worries me is, um, that like the, the core of DevOps is though we want development teams and operations teams to be like working together, right? We want them to be working in sync, we want them to be working together, we want them to be talking to each other. And, um, I kind of worry sometimes that, um, the way some IaC tools market themselves is trying to, like king, make only ops people or king make only engineers. Yeah. And, uh, putting one or the other up on a pedestal like that is kind of like antithetical to the, the core of DevOps to me. So, um, I, I would like to see people be able to use the tools that work best for them, um, regardless of whether it's like YAML or a programming language, I wish that they alongside each other more often than like mm-hmm. <affirmative> being like super divisive about it. Cause I think it's dangerous to put ops people on a pedestal or put developers on a pedestal. Cause like, it's the tech industry, Lord knows we're already all on a pedestal.

Justin Nemmers: 00:32:54

‍The, uh, so Brandt, uh, one of the things you wanna stop doing in, uh, in 2023,

Brandt Meyers: 00:33:01

‍Um, let's see. I wanna stop thinking about lift and shift <laugh> <laugh>.

Kat Cosgrove: 00:33:09

‍Yeah.

Brandt Meyers: 00:33:11

‍Um, yeah, I want to, uh, I wanna stop, um, I think stop the, the, the siloed, um, approach in, in practice. I, I think back to Kat's point, like having, having the, the ability to support the diversity within our, uh, discipline is important and that drives innovation and we should collaborate more.

Justin Nemmers: 00:33:43

‍Fantastic. And, you know, I'm actually kind of reminded in our, one of our previous chats you were talking about this kind of core continuum and like how, and this is a a little bit off topic, but I think it's relevant and interesting, so we'll bring in about it anyway, uh, it's nice thing to get to do when you're the moderator. <laugh>. So you were talking about just like open source in general and the whole kind of paradigm of open source and innersource. So like how do you, how do you take a team that is like broadly adopting new technologies and how do you, how do you turn that into that collaborative thing that, that we're all search, like really searching for, how do you turn that into a, a net gain for your team versus a, like a, either a neutral or a net loser?

Brandt Meyers: 00:34:26

‍Yeah. So yeah, there's very much this continuum of, um, something, something can change at any point within this relationship of things that are just fundamentally a framework versus, you know, some industry, um, sourced capability, um, to something that's very much intellectual property, specific to, um, a use case. And these things are all related, and companies have gotten very used to, uh, leveraging open source, um, or, you know, industry capability that's out there, um, to deliver, uh, requirements specific to product teams. But there's, there's this continuum where there's feedback across the board, right? We might, we might develop something that's maybe, um, specific to a product, digital product, and it's very much, um, relevant core to a business case, but that might impact, that might influence something that we use in the industry. And so maybe we want to, we want to suggest a change to a library or a, a reasonable component that's out there, a provider or something. And, and we need to be, we need to build that relationship back into the community to, to be able to move these things forward, both for our intellectual property and and for our industry.

Justin Nemmers: 00:35:51

‍Yeah, I think that's a really good, kind of an interesting point. I mean, I'm curious as to what you think when it comes to IaC specifically. You know, do you, do you feel like that like there even could be a lot of reuse for, for most organizations, IaC components outside of their own organizations? I mean, it feels to me that, that increasingly the infrastructure is the application, right? So it, if, if doesn't matter what cloud you're pro you're building on, you've made a bunch of infrastructure decisions and you codify them. The question is like, is that useful outside of what it is you're doing? You know, what, what do you think about that?

Brandt Meyers: 00:36:28

‍Absolutely. It's, um, you know, I, I think infrastructure as code has a huge opportunity for that. It's, we, we've started with, um, building the libraries, right? And the fundamental components that are needed to describe the configuration we need. But then, you know, we can also provide baked configurations, um, that are reusable. And those, those baked, um, you know, reusable components then can, are, are very much relevant in the industry, right? If, if you're working on some, some basic, you know, blob storage component or, or secrets management component, um, that's very relevant to everybody. And it's not quarter of your business, it's quarter industry. And so, you know, we, we talk about in our industry, we're struggling with, um, with talent and, and getting these things done, but part of the problem with our industry is we're all doing the same thing at the same time. So, you know, if, if we can focus on, as an industry figuring out how do we collectively do the same thing, uh, that's gonna be far more efficient to get things done.

Chris Short: 00:37:43

‍I would like to see more companies open sourcing what they're doing, right? Because we've gotten to this point now where we all agree infrastructure code, infrastructure as code is the right way. But it always feels like everybody, every place I've ever gone has to reinvent the wheel, right? Like after, yeah. Like,

Kat Cosgrove: 00:38:01

‍Go ahead. Why, why is it a, a secret how, like what Terraform or Pulumi code you're using to stand up the infrastructure necessary to deploy a serverless application on AWS. Like that should just be like a, a module that you feel comfortable publishing and open source, because it's not like, it's not mission critical. Nobody, nobody cares. Like, just don't publish your secrets, right? And, you know, make it clear.

Ohad Maislish: 00:38:34

‍Say, yeah. You say just,

Kat Cosgrove: 00:38:36

‍Just, just like, it's so easy. It's, it's so easy to not commit your secrets. Um, not, this

Ohad Maislish: 00:38:41

‍Is not necessarily GitHub necessarily. I, I've seen some weird things

Kat Cosgrove: 00:38:44

‍People. Yeah, for sure. Oh, it happens all the time. I used to teach at a bootcamp and like every single time there was at least one student that, that committed a secret to GitHub. It always happens, but like, seriously, like, it, it is possible to like modularize your infrastructure as code in such a way that the configuration that's application specific is just drop in and everything else shouldn't be proprietary. It's just like, would you consider it proprietary to like, I don't know, write a, an entry level tutorial with, with screenshots of how to like click through the AWS UI to stand stand up an S3 bucket? No. So why is the infrastructure as code portion not often open sourced? Right? Like

Ohad Maislish: 00:39:34

‍I have to mention that env0, we, we developed a feature that automatically tags your resources re conclusively and knows which resources are taggable and not taggable. And then the second when the developer finished that feature, he said, Hey, I think we should just, we should just open source it. Uh, it's not just for, uh, for us. Anybody can use it without being an env zero customer. And, and so we did. So I think it has And, and if that developer would not have said that,

Kat Cosgrove: 00:40:02

‍Probably wouldn't have thought of it. Well,

Ohad Maislish: 00:40:03

‍Yeah, it would not be, uh, some, some, some, uh, focus of us. Mm-hmm. So I think eventually the, the great things that happen are eventually go, from the developers, the ideas from the developers. And I think if we look now compared to five years ago, I'm sure that five years ago we would not have thought about let's open source that because it was not something that people used to do. Uh, but, but now I think it's just, it's, it is increasing. I think that trend of open sourcing some modules, some frameworks, any reusable code, I think it's a good opportunity. There is the GitHub style. Uh oh yeah, I don't remember the title GitHub, or GitHub style. I don't remember the, uh,

Kat Cosgrove: 00:40:49

‍GitHub Stars.

Ohad Maislish: 00:40:50

‍Yeah. GitHub stars. Yeah. So it's kind of, uh, an incentive now for, for developers, eh, to do those, those kind of things. And you can also, um, um, give money and donate to mm-hmm. <affirmative> to a successful developer and GitHub. And now also a company can do that and not just an individual user. So I think all in all the right things are happening in order to promote and advocate in, in that direction. But some things take, take time, I guess.

Justin Nemmers: 00:41:17

‍Yeah, I mean, it's an ecosystem thing. And we saw this, uh, in spades in the earlier days of Red Hat where we had, we would enter into a, a discussion with the company. And some of these are, are huge multinational global organizations that had active policies against open source. Now that's, this is a whole, a whole different podcast, <laugh>, this is a whole different, uh, webinar discussion.

Kat Cosgrove: 00:41:36

‍Yeah. That's just, that's a thing that feels very legacy to me though. That's just like a absolutely a big old company vibe. Like any big old company. I expect, like getting them involved in open source and an authentic way that doesn't involve, like, fighting with legal for six months. I expect that to be a battle with any like sufficiently large, sufficiently old company or with like, literally any company in some specific industries, like, like banking, right? Mm-hmm. <affirmative> like in that space. I expect it to be a, a fight every time. But, you know, we're seeing more and more companies with OSPOs, uh, open source program offices for those listening who do not, uh, know what an OSPO is. And I know more and more authentic participation. So I, I agree. I think the, the tide is turning.

Justin Nemmers: 00:42:20

‍Yes. Ohad, I think that we've struck a nerve, and that would be a really interesting topic for one of your, uh, future infrastructures, code, uh, podcasts.

Chris Short: 00:42:28

‍You can sign me up for that if you want. Yep. I'm also in the

Justin Nemmers: 00:42:31

‍Meantime, um, Let's, uh, let's try to, to reign this back in a little bit. Now, we'd started to talk about this kind of at the, the head of this discussion. Uh, and the good news is we do have a bunch of, uh, excellent questions. So we'll try to kind of keep this last question a bit a bit concise, but, uh, so we talked about, you know, what's upcoming, what we think we're gonna see less of, um, in 2023. And now, you know, it, it bears without saying that there's, there's a lot of turmoil in the industry, right? So at the end of 2023, uh, excuse me, at the end of 2022 <laugh>, uh, we've seen just a, a tremendous, and, and certainly continuing into 2023, tremendous number of, uh, of large and wide reaching layoffs, right? So, you know, the, the question is, is it, do we, is this having a real and direct impact on your teams today? Uh, and then how do you, how do you see it affecting teams that, that really are already stretched pretty thin? I mean, that was the, the story of automation back in the day was that some people were afraid of automation because they thought that if, well, if I automate my job, like I'm not gonna have a job. Uh, Chris, you hit the nail on the head earlier that, that it doesn't work that way, right? Like, in the end, there's always gonna be enough work, there's always gonna be more work than we can reasonably get done, even with automation. So we can kind of push that, that part aside, but it, um, it, it seems kind of like a weird paradox. So we're starting 2023 with a, a reduced workforce. The expectations and requirements are continuing to increase. Like what happens? We're gonna have to do more with less. Like how, how does that look? How, how do teams adjust to that?

Chris Short: 00:44:06

‍I think you hit the nail on the head, right? Like, do more with less. And that is something that, you know, I've been telling people we have to do with all manner of technology. Um, not because I think we need to eliminate headcount or anything like that. I think it's because we need consistency. It's because we need the things that IaC brings. And the more people that are doing that, the better. Now, like to folks that got laid off, I really feel bad for you. I would say, like, if you're a developer, learn a little bit about infrastructure while you're, you know, spending your time applying for new jobs or whatever it is you intend to do. And vice versa, if you're an infrastructure person, spend some time picking up some code, or at least get familiar with GitHub

Kat Cosgrove: 00:44:56

‍And pick up some Python. Yeah. You know, there's like, Python is superhuman, readable. Just, just pick up some Python.

Chris Short: 00:45:02

‍Yeah. And the more that we have crossover like that, the better things we'll get. Because I've worked with the large development teams and they don't necessarily know anything about infrastructure. So sometimes they do commit a secret to GitHub or something to that effect where it's definitely damaging to the company, and it's just because they didn't know better. And like I said, information's gonna be harder to come by. So we have to share these best practices more.

Justin Nemmers: 00:45:34

‍So, so Brandt, let me ask you this. In a, as a, as a, a leader in a, uh, business oriented, kind of consumer oriented industry, you clearly have many more demands on your team than you have ability to, to rapidly solve those problems. So I guess, yeah, how, what are some of the things you're doing today in order to do, to do less with or do more with less? That almost sounded backwards. Do less with more. Yeah. <laugh>, so basically like you have a bunch of requirements and you have a, you have a limited team no matter how, how big the funding looks like. What, what are some of the things that you're doing to, to assist with that?

Brandt Meyers: 00:46:13

‍So our, our automation approach is community automation. That's, that's the, the name of the program. And so I think it goes back to influencing the practice of engineering that, uh, we, we can work together and leverage, um, reusable components that, uh, that all teams need, you know, and, and that helps accelerate, um, the engineering process. Um, yeah. And, and it's, and and rethinking about what, what's our approach? Where's our opportunity? Um, there, there's some thoughts even around, you know, I see, I've seen some recent articles about repatriation. I think that was a big topic in 2020. Um, but that's coming up again for cost, right? So looking at, um, ways to, um, provide value, provide a, a cost effective, um, solution, something that's more efficient, uh, helps with, uh, with delivering on the, the business need.

Justin Nemmers: 00:47:22

‍Absolutely. Ohad, what do you think, I mean, you, you ultimately lead a, a large engineering team that, uh, we always have more feature requests and things that need to get fixed and tech debt and all of that. What, what do what do you see happen here?

Ohad Maislish: 00:47:38

‍Definitely there are some, uh, obviously there are tons of layoffs recently. Uh, but the first thing, uh, that, that I always think in the last few months is that it's, it's, it's not really layoffs. It's more like a correction. If we look at two or three years back, all in all technology, uh, has grown dramatically, probably too much, uh, a year ago. And so far it's still, uh, a correction. But I think in any case, the, in order to be successful, as we mentioned earlier, you need to do more with less. You, you hear me very often say Justin, uh, the phrase lean and mean, and I really mean that is, uh, is the DNA of, of every company should, uh, persuade and specifically about, about engineers. As, as Chris and Kat mentioned earlier, I think engineer with skills 10 years ago, like 30 years ago, compared to 20 years ago, 20 years ago, compared to 10 years ago or now compared to 10 years ago, even five years ago. You cannot assume that the knowledge and the experience that you have is, is still relevant. And you always need to, to think about your next, next steps and how you, how you improve yourself. And that's maybe why the engineers at env zero make fun of me because they say that I'm, I'm no longer a good engineer because my knowledge is somewhere a bit stuck five or 10 years ago when they really stopped programming hands on. And I think they're really, uh, they have a point. So I think every engineer should realize what they do best. If it's more backend or front end or mobile or kernel or infrastructure, what they like doing and keep improving themselves in, in that direction, um, they, they, they should do well. And for organizations always try to optimize. And if somebody becomes less relevant, you need to make the difficult decision and, and optimize your organization accordingly. You cannot assume that the things that work well three years ago should continue to work well now.

Justin Nemmers: 00:49:52

‍Fantastic. Uh, alright. So we are I think, rapidly approaching some, some Q&A and we've got a lot of good questions queued up. Uh, any, any parting comments from, um, from the panel here on, uh, on that last point about doing more with less?

Brandt Meyers: 00:50:09

‍I'll say, I'll say one thing. The other side of it is, um, it's exciting to be a technologist. It always has been. Um, and it still is. And so there's always opportunity, um, even, even with, um, things that can be discouraging. There's always opportunity in our field and that's, that's a pretty cool thing to be a part of.

Kat Cosgrove: 00:50:29

‍Um, I, I will say that I think that some, um, startups are about to have access to some incredible talent that they might not have had access to otherwise with the number of people that large tech companies have laid off that have a decade or more of experience with some of the most cutting edge technology the world has ever seen. Um, and that's, that's fantastic for those startups. But I will say that, um, watching things like this happen at such a large scale is, um, emotionally difficult even if you're not impacted by it yourself. So it's important to, you know, watch your, watch your own attitude and like, be be careful about are you okay? Right? And times like this also unfortunately, give us a very harsh reminder that our employers are not our friends and they are not our families. Even if they say we're a family here, no, we're not because they like the, these kinds of things do happen. We are watching it happen. And that is, that is upsetting. So it's a good thing to remember so that maybe if you are made redundant and laid off, it is a little bit less emotionally devastating. Maybe it doesn't feel as much like a, like your dad kicking you outta the house or whatever, right? Um, a business decision was made. And, um, so that is also why I don't like it when employers pulled a, uh, wear a family here card. It feels manipulative in the face of things like this happening. But, you know, this is difficult for all of us to watch and all of us to be involved in. So take care of yourself. You know,

Ohad Maislish: 00:52:04

‍I fully agree and I want to give another example that I personally, uh, uh, face from target time when investors, uh, pitch me and they sometimes say we're like family or we're in the same boat. But, but even, but eventually it's, you want to be as close as possible. You wanna work together, you want to make everybody feel well, but eventually it's not the exact same situation. Yeah. In, our case, uh, investors have preferred stocks, they have voting rights that we founders don't have, and

Ohad Maislish: 00:52:42

‍It’s okay. The structure is okay, but let's not fool ourselves that, uh, it's the same exact same thing as, uh, as a family. Again, everybody has, or almost everybody has good intentions and we're trying to, to do our best. But I I, I fully agree with you, Kat, eventually it's, uh, it's not, it's not the same thing.

Justin Nemmers: 00:53:08

‍Awesome. Um, alright friends, so we do have a number of good questions. Uh, and some, I think humorous quips. So, so Basil initially, uh, suggested that, uh, we should have a, a Terraform provider for Windows, uh, to which I, I cheekly responded, pull requests likely welcome <laugh>, I dunno, we accepted, but, cause it's not my, you know, Hashi not your goal. None. Yeah, you can certainly always try. Uh, but let's kinda dive in. I think there's some really interesting ones here about, uh, just IaC and, and I think we, we struck a bit of a nerve talking about like what is IaC versus not a IaC. So, so Basil continues and he says, why would you consider config management tool as part of iac? From what I've seen, there are always teams who do iac ie. Provision infrastructure to level of VMs, and then a separate team who actually does VM configuration using something like Ansible puppet, et cetera. So what, um, what are some of the thoughts that, uh, you have on that one? I'm actually, um, primarily interested in branch's thoughts on that. So how, how do you guys handle it?

Brandt Meyers: 00:54:13

‍Um, it's a good question. So we do have, uh, Ansible or maintaining configuration. I think runtime configuration is separate from IaC, um, not to say you can't do it, right, but it's <laugh> there. There's, I think it goes back to Kat's point of there are, there are tools that are specific to use cases and they're, they, they all have their benefits and so making sure that you have a robust toolbox that has, that can accommodate the capabilities that you need.

Justin Nemmers: 00:54:51

‍Yeah, I mean, it comes down to kind of using the best tool for the, for the job ultimately. Mm-hmm.

Chris Short: 00:54:55 Yeah. I think I, I really wanna see the, we're a Terraform shop or we're an Ansible shop, or we're a one tool shop kind of die this year because you can't, it's just not feasible.

Kat Cosgrove: 00:55:06

‍Yeah. I mean, like, you can do it, but it's not like, it's not the best way to do it. No. Right. Like, it, it does introduce some difficulties, but I think that was in, uh, response to my assertion that, um, configuration management is a type of infrastructures code, um, to clarify that we only had configuration management for a long time. That is what we called it. But we were automating things in a code like way, like if you, yeah, you can very easily make an argument that a make file is infrastructure as code, that that's configuration management. It's not, not a, I mean, you gotta squint at it a little bit, but you can 100% make that argument <laugh>. But, um, infrastructure as code is just like now an umbrella term that happens to include configuration management and configuration management tools are now starting to do things in a more code like way. Um, so it's, it's kind of just a semantics thing at that point.

Justin Nemmers: 00:56:10

‍No, I mean, and so this is actually interesting. I think, um, a, uh, a follow on to that and I think it's, it's appropriate for you again, Kat is, uh, where's cross plane? So where's Crossplane in this entire equation? You know, we did the, we did the poll, uh, looks like some, some folks had responded about, uh, about Crossplane, um, but not many. I think one, one person said that they were either using or looking at that. So how, how do you feel like this fits into the overall mix here?

Kat Cosgrove: 00:56:37

‍You know, I don't actually hear about it all that often. Um, and I only hear about it in the context of shops that are like fully 100% cloud native, like, that's, that's about the only time I ever hear of cross plane being used. Um, I have never worked in a like fully cloud native environment, so I've never actually used Crossplane myself. Um, however the people I know who do use it are like super passionate about it being like the one and only true way to, uh, to do what it does. Oh yeah. It's like may as well be like from on high at this point. Um, it's community is pretty, pretty big and active. A friend of mine, um, used to be a, uh, a Crossplane maintainer and so I, I've never used it personally, but I only hear about it in like a hundred percent cloud native shops.

Chris Short: 00:57:30

‍Yeah. I think that's a fair assessment, Kat. I think a lot of teams that are spinning up cloud native resources are saying, oh, Crossplane’s probably are Terraform here. Yeah. Um, pretty much. But yeah, it's, I, I hear about it at KubeCon. I hear about it at the occasional, you know, customer site, but it's, it's few and far between right now. But yeah, they are trying to like expand their breath outside that Kubernetes world,

Kat Cosgrove: 00:57:55

‍Right? Like, I think it'll, it'll, I think it'll gain traction. I don't think we're gonna see it like disappear and it's entirety, but it is certainly not going to and, uh, as popular as like Terraform,

Chris Short: 00:58:06

‍Right? No, I don't, I don't think it's, it's, yeah. I mean, unless it does some magic trick here soon.

Ohad Maislish: 00:58:12

I fully agree and I think I have a, I have my own explanation of whats, what's happening here. Most of the DevOps engineers, not most, but a lot of the DevOps engineers I talk to say things similar to what Kat reference of something like CDK and, and Pulumi. So they prefer Pulumi over Terraform if it was just about, you know, writing code, writing the infrastructure as code, but then when they're looking at the, uh, overall solution that they need to provide to, to their companies and, uh, we are using also, uh, Auth0, so we need, uh, to work with that. Mm-hmm. And where is the Pulumi provider for Auth0?

Ohad Maislish: 00:59:00

‍I think one of the reasons that Terraform is doing much better than Pulumi, it's not because it's a better framework, but it's more about the ecosystem and the timing. The timing of when Terraform started in great work by HashiCorp, obviously when they started, um, educating the market in that direction, later came Pulumiand later came Crossplane. So I think Crossplane their main, uh, issue is not the technology is the time to time when they enter the market. Why now slide for if you're familiar with investors who's a pitch

Ohad Maislish: 00:59:37

‍Yeah. It's, it's why now? Why start Crossplane now when the Terraform is already, uh, with such strong partnerships with so many, uh, yeah. vendors

Kat Cosgrove: 00:59:48

‍And I, I think you have to consider the, like the applications community to be part of the ecosystem because the, the community around something like Terraform with that maturity is a super valuable resource to your engineers because when they run into a problem, they're gonna run into a problem, right? When they run into a problem with Terraform, that issue is Googleable. Somebody else has run into that problem and somebody else has documented the solution

Ohad Maislish: 01:00:14

‍I think, I think it's way more than Googleable Googleable, uh, it's not, it's not just that. Let's, let's talk for a moment about what Terraform provides with site Terraform the, uh, the framework. So you need to have policy as code boom, OPA, okay? And you have policy as code for Terraform, but not for Pulumi. Crossplane. You need some, uh, uh, static

Kat Cosgrove: 01:00:35

‍Pulumi does have a policy is called code tool, it's called crosswalk. But uh,

Ohad Maislish: 01:00:39

‍Is it just the Pulumi, is it just for the Pulumi service or is it open source? Like open policy agent?

Kat Cosgrove: 01:00:46

‍It is not open source, I think. Okay.

Ohad Maislish: 01:00:48

‍So, so that's, as you mentioned, as you mentioned,

Kat Cosgrove: 01:00:51

‍I don't work there anymore, so I'm not sure.

Ohad Maislish: 01:00:53

‍Open, open Policy agent is the, the defacto start out today for, for policy code and it works very well with, with Terraform. And let's talk about the security and static analysis. You have Checkov, you have Terrascan, you have TFsec, uh, you have Kicks. All of those work very well with Terraform. If you look at cost, uh, you have Infracost that does the uh, cost estimation for, uh, for core request you have Docs, automatic Docs. For, for Terraform you have TF Flint. Uh, you have so many other great things that you can use. Um, and even in env0, we started with support for Terraform. Only later on we added a support for Pulumi and uh, and CloudFormation and others because it makes sense to focus, well, both of the market is, uh, what most of the market is using. So Terraform and its, uh, peripheral tools is super powerful compared to just Crossplane.

Kat Cosgrove: 01:01:56

‍So cross guard, I was, uh, I was initially wrong. Crossguard, um, is Pulumi policies code tool, and it is open source. Um, so, but it looks, uh, looks like they added, um, providers for other cloud providers after I left, cuz it, it now does support things other than AWS.

Ohad Maislish: 01:02:13

‍Oh, awesome, glad, yeah, glad to hear. But I can assume that open policy agent has much bigger community

Kat Cosgrove: 01:02:19

‍Than, oh, yeah. OPA is almost certainly like way more mature. Way more mature. Um, and also like, I'm just like functionally more familiar with opa, so mm-hmm.That's, that's still like probably what I would use, um, just due to being more, more familiar with it.

Justin Nemmers: 01:02:38

‍Fantastic. All right. So I'm gonna do one last, it's kind of a question, kind of a comment. Uh, and then we'll go ahead and wrap this up as we are a couple minutes over here. Uh, Diego asks, how do you see the specific services like provision, orchestration, configuration today? I see the Terraform Ansible working together, uh, impossible for only one platform to provide all modern infrastructure services like provisioning, cell healing, continuous compliances, code, uh, and others. So I think that actually summarizes it quite nicely. Uh, any, any kind of party comments on, uh, on that one from the, the panel here?

Chris Short: 01:03:13

‍I think policy is gonna become a big thing this year, right? Like gone are the days where you're allowed to have a S3 bucket that gets kicked over or compromised somehow, right?

Chris Short: 01:03:27

‍AWS does a proactive job and we're changing the defaults on S3 now in April, I think, to make it so you can't, like the default is not going to be publicly exposed to ever

Chris Short: 01:03:40

‍And that I think is, you know, it's overdue, but it's a hard change to make. Yeah. For as many customers as we have. So addressing that, and then, you know, companies like AWS, IBM, Microsoft, you know, Google, we're gonna have to get a lot better at handing people good policy, right? Yeah. Like, oh, you're using this service, this is the policy we recommend and here's why. Right? Like, we need to develop more materials around that specifically.

Kat Cosgrove: 01:04:12

‍Yeah. I would like to see, um, more, more companies not, not trying to be like a pocket multi-tool, right? Like, because that, like Diego is right there, there isn't like, there, there isn't one tool that is like the best thing at all of these, like very different, very specific things, right? So maybe stop trying to be a multi-tool cause like you're really good at the one thing, but then you're like kind of mediocre to actively not good at all of this other stuff. And then trying to lock people into using like what is in aggregate a subpar solution at that point. So, uh, specialize in what you're good at and, you know, actively collaborate with other tools that fill in the gaps that are really good at provisioning or are really good at policy instead of like trying, trying to force it, you know? Um, and if, if you are a company that is looking into the use of these tools, again, don't, don't try to use a multi-tool just because it's really good at one thing. If it's really bad at all of the other stuff that you need to, like, it's, it's okay to have more than one solution. Sometimes. There, there is a way to smooth out the friction there. You, you really do not having one tool that is, is bad at most of what it does, but hey, it's only one tool that also introduces a ton of problems.

Justin Nemmers: 01:05:39

‍Yeah. This is great. So, I mean, I think that that actually is a great summation because in that little equip, Kat, you, you touched on a lot of points that we talked about today. Uh, how do you get team members up and functional? How do you, uh, effectively collaborate by, by kind of using the right tool for the right job? Um, how do you have a culture of innovation that enables you to, um, to rapidly adopt these new tools and actually do real things with them that are still backed by policy on some level? Uh, all of which are, are certainly, I think we can all agree will become more and more important as, um, as IaC adoption, uh, continues to grow within, within organizations. So with that, uh, I want to thank everyone for joining us today. Um, now this recording will be made available to all of the registrants, so we will be emailing you a link to it. You can download it and then I'm sure we'll do lots of little, uh, clips and snippets from, from here with some nice, uh, some nice points on 'em. But, um, there are some existing questions that we did not get to. I apologize for running out of time on that. However, if, uh, they're ones that do necessitate a direct answer, we'll be more than happy to provide those via email after the fact. Uh, with that, thank you very much. Have a wonderful rest of your day and a special thank you to, to all of our panelists.

‍

Recap: 2023 Infrastructure as Code Roundtable

Ohad Maislish

Co-founder of env0 & OpenTofu

Blog

read time

min

Configuration as Code helps teams manage infrastructure efficiently by automating repetitive tasks and improving reliability. However, it also brings new challenges—managing secrets securely is one of the most critical. Without proper handling, sensitive data like API keys, passwords, and certificates can be exposed, creating security risks.

Protecting secrets while maintaining automation benefits is essential. For instance, expecting operators to manually input secrets during an automated process is both impractical and error-prone. In this post, we explore Ansible Vault, a powerful tool that secures sensitive data without disrupting DevOps workflows.

What is Ansible Vault

Ansible Vault is an encryption tool included with Ansible that protects secrets while enabling DevOps workflows. In this article, we will take a comprehensive look at Ansible Vault to understand its use cases and features.

Ansible Vault is a utility included in Ansible that can encrypt and decrypt arbitrary data using a password. It can encrypt a variety of data using AES256 encryption, including:

Structured YAML files, such as variable files or even entire playbooks
Configuration files with sensitive information
Individual variables in Ansible playbooks

Most importantly, Ansible Vault integrates transparently with other Ansible commands, such as [.code]ansible[.code] and [.code]ansible-playbook[.code]. These commands can automatically detect and decrypt encrypted data to use in standard Ansible workflows.

For example, an Ansible playbook can reference variables stored in an encrypted variable file. These files will automatically be decrypted at runtime using the appropriate password.

Password Protection and Encryption

Creating an Encrypted Files

At its most basic level, Ansible Vault can encrypt entire files with a password. For example, consider a situation where you want to store an API key as a variable in a YAML file. You can create the initial vault encrypted file using [.code]ansible-vault create[.code]:

The file will launch in your shell’s default editor, as controlled by the [.code]EDITOR[.code] environment variable. If no editor is set, it will default to [.code]vi[.code]. Create the file like any other text file and save it:

Now, when you try to view the file, notice that it is an encrypted blob:

Viewing and Editing Encrypted Files

You will frequently need to view the contents of vault-encrypted files or edit them directly. You can do this with the [.code]view[.code] and [.code]edit[.code] commands.

The [.code]view[.code] command displays the contents of the encrypted file:

The [.code]edit[.code] command launches an editor to modify the encrypted file using the shell’s default editor:

Decrypting an Encrypted File

Sometimes, you may want to fully decrypt a file. For example, you may determine that the data is no longer sensitive and doesn’t need to be protected as a secret.

You can fully decrypt a file using the decrypt command. This command fully removes the encrypted file and leaves only the decrypted file in place:

Encrypting an Existing File

Configuration as Code is frequently used to transfer sensitive configuration files to remote hosts. Ansible Vault can fully encrypt an existing file. For example, consider a configuration file with sensitive information in it:

Ansible Vault can encrypt this entire file using a password with the [.code]encrypt[.code] command:

Changing the Encryption Key

It’s a security best practice to regularly rotate encryption material. This principle also applies to the passwords used to protect files encrypted with Ansible Vault.

Ansible Vault makes it easy to rotate the encryption key for a file using the [.code]rekey[.code] command. Simply provide it with the existing password and a new password:

Encrypting Variables in a Playbook

The examples we have looked at so far encrypt entire files. This is the most common way to use Ansible Vault. However, there are also situations where you want to encrypt data within a playbook while leaving the rest of the playbook unencrypted.

Ansible Vault enables this pattern with the [.code]encrypt_string[.code] command. You can use [.code]encrypt_string[.code] to encrypt the contents of an arbitrary string and then place these contents in a playbook.

For example, consider a playbook that makes an HTTP request to a password-protected API endpoint:

We want to protect this password using Ansible Vault, but we don’t want to encrypt the entire file. You can use [.code]encrypt_string[.code] to encrypt the API key:

Notice that [.code]encrypt_string[.code] allows you to directly input the string you want to encrypt into the terminal. You end the string with [.code]CTRL+D[.code], not with a newline character. This is important to remember, as any newlines you enter into the terminal will become part of the encrypted string.

Finally, you can insert this encrypted string directly into the playbook:

This approach is useful for very basic playbooks, but it has limitations. You must encrypt each string individually, which can be tedious and time-consuming. Additionally, there is no way to easily rekey all of the encrypted strings in a file. Instead, you must re-encrypt each string.

A better approach in most scenarios is to use a fully encrypted variable file and limit the use of [.code]encrypt_string[.code]. However, using [.code]encrypt_string[.code] can be helpful in very simple playbooks that don’t require the overhead of fully encrypted variable files.

Running Ansible Plays with Encrypted Files

By now, you should have a good understanding of how to use Ansible Vault to create and manipulate encrypted files and data. Ansible integrates transparently with Ansible Vault and allows you to use encrypted files and variables within your plays. Ansible automatically decrypts the encrypted data using the password that you provide.

To illustrate these principles, you can use a playbook that contains both encrypted variables and fully encrypted files:

This playbook performs two tasks:

Makes an HTTP request using the encrypted [.code]api_key[.code] variable
Copies an encrypted configuration file to the remote host. This configuration file is created using [.code]ansible-vault create[.code]

Next, it’s time to run the playbook. Ansible will automatically recognize data encrypted by Ansible Vault and decrypt it at the appropriate time. Ansible relies on a password to make this work. There are three main methods for providing Ansible with a password for decrypting protected data:

Provide the password manually via the command line using [.code]--ask-vault-pass[.code]. This is inappropriate for automated scenarios but works well when testing.
Reference a password file that contains the password with [.code]--vault-password-file[.code]. This is the most common approach. This file should be carefully locked down to prevent exposing the password.
Look up the password using a script specified by [.code]--vault-password-file[.code]. This is an advanced approach that is very useful when you want Ansible to interact with an external secret storage system, such as AWS Secrets Manager.

For this example, we will use a password file:

With everything in place, it’s time to run Ansible:

Notice that the [.code]ansible-playbook[.code] command transparently decrypts the necessary files and variables. This makes it easy to begin using encrypted secrets without disrupting existing automation workflows.

Advanced Use Cases

The basic features we have covered so far are enough for most scenarios. However, Ansible Vault also features several advanced usage patterns that are helpful for more complex environments.

Managing Multiple Vaults

Large environments frequently use multiple secrets with different permission levels. For example, a different set of secrets may be used for staging and production infrastructure. Operators may also choose to encrypt different files and variables using separate passwords.

Ansible Vault allows operators to work with multiple vaults, each uniquely identified by a vault ID. Vault IDs provide a “hint” to indicate the correct password to use when decrypting a vault file.

For example, consider a situation where you want to encrypt two different configuration files with different passwords. Ansible Vault enables this with the [.code]--vault-id[.code] flag. This flag takes its argument in the format of ID@SOURCE, where “ID” is the vault ID to use, and “SOURCE” is the location to find the vault password.

For example, you can encrypt two different files with different passwords provided via the command line prompt:

Next, you can tell [.code]ansible-playbook[.code] about the appropriate place to obtain the password for each vault using the [.code]--vault-id[.code] flag. Notice that the password for the vault with ID “config1” is given at the prompt, while the password for the vault with ID “config2” is provided through a password file:

This approach provides great flexibility when using multiple passwords. However, it can quickly become complicated. You should still try to maintain simplicity when designing your password strategy.

It’s important to understand that vault IDs are just hints. Vault ID matching is not strictly enforced by Ansible unless you set the [.codeDEFAULT_VAULT_ID_MATCH[.code] environment variable. Ansible will try all provided passwords with all provided vaults until it succeeds or fails to decrypt a vault.

Integrating with a Secrets Manager

Storing an Ansible Vault password in a text file or entering it in the command line is appropriate for basic use cases, but advanced environments will typically use an external secrets storage solution. For example, your environment might use HashiCorp Vault, Amazon Web Service Secrets Manager, or your in-house solution.

Ansible makes it very easy to look up the decryption password for a vault with a client script. Client scripts can perform whatever logic is necessary to look up a vault’s password, including interacting with external secret managers. The script then prints the password to standard output, and Ansible uses this password to decrypt the Vault.

The example below uses the [.code]aws-secrets-manager-client.sh[.code] script to look up a vault password. The actual content and logic of this script isn’t important; all that matters is that the script prints the password to standard output for Ansible to use:

Using a client script provides the best of both worlds: You can encrypt files and variables in your Ansible playbooks and Configuration as Code repositories, and also integrate with external secrets managers to keep your Ansible Vault passwords safe.

Practical Tips

Version Control

A significant benefit of the Ansible Vault approach is the ability to store encrypted files and variables directly in your version control system. This avoids the need to store data in multiple places, providing you with a single source of truth for your Configuration as Code.

However, you must still ensure that any sensitive information, such as the password file for Ansible Vault, is stored outside of the version control system.

Environment Variables and Ansible Configuration

There are several Ansible configuration directives related to Ansible Vault. While you don’t need to know all of them, it is helpful to mention the most common directives that you will encounter:

[.code]DEFAULT_VAULT_ID_MATCH[.code]: This environment variable controls vault ID matching. By default, Ansible will not enforce strict ID matching and will try every password with all vaults. Set this variable to “True” to change this behavior.
[.code]DEFAULT_VAULT_PASSWORD_FILE[.code]: This environment variable specifies the default vault password file.
[.code]DEFAULT_VAULT_IDENTITY_LIST[.code]: This environment variable is equivalent to specifying multiple [.code]--vault-id[.code] flags, and it can be useful for shortening the length of your [.code]ansible-playbook[.code] commands.

Understanding When Files Are Decrypted

Ansible will decrypt files as needed when running plays, and the files will remain encrypted at rest once the play has been completed. However, files will be decrypted at rest on a target host when they are used as the [.code]src[.code] argument to the copy, template, unarchive, script, or assemble module.

This is intended and desirable behavior. It allows you to decrypt a file and place it on a remote host. For example, you can reference an encrypted configuration file in Ansible’s copy module and it will be decrypted and placed onto a remote host.

While this behavior may seem obvious, it's important to understand the scenarios when Ansible will leave a file decrypted at rest.

Integrating Ansible with env0

env zero includes native support for Ansible, enabling you to use your existing playbooks alongside its infrastructure lifecycle management capabilities. With Ansible templates, you can consistently deploy environments while leveraging env0's features like controlled access, cost estimation, and automated deployment flows. Learn more here.

Final Thoughts

Protecting sensitive data while still enabling Configuration as Code best practices is a challenge for DevOps teams of any size. It is one of the earliest challenges faced by organizations when they automate their configuration management practices, and it continues to challenge mature teams. A robust approach must be flexible enough to preserve velocity without compromising on security.

Ansible Vault is an ideal solution for teams that already leverage Ansible in their automation workflows. It has a very low entry barrier, and its basic and advanced features make it appropriate for a variety of scenarios. Simple environments can benefit from encrypted vaults with basic password authentication. Advanced environments with complex needs can tier their vaults using vault IDs and externally stored vault passwords with frequent rotation.

Ansible Vault is a simple utility that offers robust secrets protection in a variety of scenarios. It is an important component of any Ansible environment’s security posture.

Frequently Asked Questions

Q. What is Ansible Vault?

Ansible Vault is a utility for encrypting secrets. Secrets can include variables inside Ansible playbooks, external variable files, or even arbitrary data. Ansible Vault integrates transparently with other Ansible tools, such as the [.code]ansible-playbook[.code] command. These Ansible tools can automatically decrypt and use secrets in playbooks and Ansible commands.

Q. Is Ansible Vault just for encrypting Ansible playbooks?

No. Ansible Vault can also encrypt arbitrary files, such as sensitive configuration files. Ansible can automatically decrypt these files as necessary, such as when they are transferred to a remote host.

Q. How is Ansible Vault different from an external secrets manager, such as AWS Secrets Manager?

Ansible Vault is built directly into Ansible and doesn’t require any additional modules or external infrastructure to work. It directly encrypts files using a shared key. External secrets managers exist outside of Ansible and require their own configuration, tooling, and modules to work with Ansible.

Q. Can you integrate Ansible Vault with an external secrets manager?

Yes. Ansible Vault uses a shared secret password to encrypt and decrypt secrets. This password can be stored in an external secrets manager. Ansible can look up this password in an external secrets manager at runtime using a client script.

Protect Secrets and Passwords with Ansible Vault: A Practical Guide with Examples

Anthony Critelli

Developer Relations

News

read time

min

OpenTF Forks Terraform, Insists HashiCorp is the Splinter Group

Related Content

‍Infrastructure-as-code: Why IaC Matters for Cloud Provisioning

Infrastructure code: Key Benefits of IaC

‍Terraform CLI IaC Tool (Terraform Code and State)

Get Started

1. Install Terraform

2. Write your first Terraform Configuration

3. Run Terraform Workflow (Terraform CLI in Action)

‍OpenTofu IaC Tool: Open Source Terraform Alternative

‍Key features:

‍Get Started

1. Install OpenTofu

2. Define your OpenTofu configuration

3. Run OpenTofu workflow

Pulumi IaC: Infrastructure-as-code Tools for Developers

Get Started

1. Install Pulumi and create a project

2. Write a script to provision infrastructure

3. Deploy Stack

Crossplane IaC Tool for Kubernetes

Get Started

Ansible IaC Software for Configuration Management

Get Started

Salt IaC Tools for Automation

Get Started

Chef IaC Tool for Infrastructure Automation

Get Started

Puppet IaC Tools for Infrastructure Management

Get Started

Alternative to Terraform: What Usually Triggers a Switch

Frequently Asked Questions/FAQs

Q. What is the difference between Pulumi, Crossplane, OpenTofu, and Terraform?

Q. Which is better: Terraform or Ansible?

Q. What is the difference between Chef, Puppet and Ansible?

Q. What tools are competitors of Terraform?

Top Infrastructure as Code Tools and Terraform Alternatives

Introduction

Overview of Ansible and its Role in Automation

Importance of Ansible Conditionals in Playbooks

Purpose of this Guide

Understanding Ansible Conditional Statements with Ansible "when" Statements

What Are Conditional Statements in Ansible?

Basic Syntax of Conditional Statements Using the "when" Keyword in Ansible

Example: Simple "when" Statement

Key Points About the "when" Statement

Using Ansible Conditionals with Multiple Conditions

Combining Multiple Conditions

Logical Operators in Ansible:

Example: Combining Conditions

Using "or" for Alternatives

Practical Applications of Multiple Conditions

Environment-Specific Automation

Conditional Debugging or Logging

Tips for Combining Conditions

Working with Ansible Facts

Conditionals Based on ansible_facts

Example: Using ansible_facts

Examples of Common Facts

Practical Use:

Using Facts to Select Variables, Files, or Templates

Example: Using Facts to Load Different Packages

Example: File Selection Based on Facts

Loading Custom Facts for Conditional Use

Example: Using a Custom Fact

Using Ansible Register and Conditionals Based on Previous Tasks

Using Output from Previous Tasks

Example: Capturing Task Output with register

Creating Dependencies with Registered Variables

Example: Using Registered Variables for Dependencies

Integrating Ansible when Fail

Tasks That Only Run on Failure

Example: Handling Task Failures

Practical Example: Reusing Registered Outputs

Example:

Integrating Conditionals in Loops

Using Conditionals with Loops

Example: Applying a Task Conditionally Within a Loop

Practical Examples

Example 1: Filtering Items in a Loop