Infrastructure as Code Is Dead: Long Live Infrastructure from Code

As we’ve modernized computing at scale, configuration management tools arose to simplify and standardize infrastructure management. As organizations moved to the cloud, the provisioning and managing requirements exploded exponentially, making previously manual tasks almost impossible. DevOps was coined to help implement the automation best practices we learned from software into infrastructure practices.

The cloud shift and the rise of DevOps introduced Infrastructure as Code (IaC), allowing us to configure and manage infrastructure like how we manage our software. Unfortunately, IaC hasn’t quite lived up to its promise to easily deploy and maintain cloud resources quickly.

The Rise and Fall of IaC

Infrastructure as Code originated in the mid-2000s, as we saw Amazon Web Services (AWS) release the first version of Amazon Elastic Compute Cloud (Amazon EC2) in 2006, and Puppet and Chef introduce declarative, domain-specific languages (DSLs) to help system administrators automate configuration and provisioning in 2005 and 2009, respectively. Suddenly, everyone could use as much computing resources as they needed, and new tools were required to manage their new, more expansive, infrastructure.

When containers, microservices and Kubernetes became more widely used in the mid- to late-2010s, configuration management needs changed again. The same year Kubernetes was released, HashiCorp’s Terraform was also released, offering a new way to standardize cloud provisioning and coining the phrase “Infrastructure as Code.”

From the beginning, configuration management tools increased the efficiency of configuring and deploying infrastructure, enabling faster deployments through streamlined processes and automated workflows. By automating tasks and minimizing manual errors, IaC reduced the operational overhead associated with infrastructure management, and reduced the likelihood of errors that occur with manual management.

The hope was that, because IaC incorporated best code practices, developers would be able to work alongside system administrators and other infrastructure engineers to deploy code effortlessly at scale, enabling closer collaboration between development and operations teams.

But just as IaC expanded the ability to deploy to the cloud, it added complexity to that deployment by combining teams with different experiences and expertise and requiring them to find new ways to work together.

The Challenges with IaC

Despite the clear benefits to scale and automation that come with IaC, it remains very complex because cloud infrastructure is complex and constantly changing. As more teams are involved with cloud provisioning, they have to agree how best to use IaC tools and learn the nuances of each one they choose. With these added pressures, fresh solutions promising to improve the developer experience without increasing risk are emerging. To create the next generation of solutions, organizations need to understand where the problems truly lie for development, platform engineering and security teams.

Learning Hurdles

With multiple tools and frameworks to choose from, learning new languages and tools can be difficult for teams whose experience stems from manual infrastructure provisioning or writing application code. In addition to requiring a new programming language and interface, most IaC tools define and support infrastructure and resource management using declarative languages.

That means teams must learn how to define the desired state of the infrastructure environment rather than outlining the steps required to achieve a result, a challenge for new and experienced programmers alike.

Plus, each IaC tool excels in different categories, often resulting in the use of multiple tools to manage infrastructure effectively. Having more than one tool can result in more complexity and compatibility issues, and using multiple tools doesn’t eliminate the need for infrastructure expertise.

Bottlenecks and Inefficiencies

Contrary to the goal, IaC can slow down the software development life cycle as diverse teams work to provision application cloud resources. Different stakeholders must master multiple IaC tools and languages, manage complex infrastructure with intricate dependencies between resources and have the specialized knowledge required to securely implement IaC. Platform engineers are overwhelmed with deployment requests and must comb through infrastructure configuration files to identify issues that could result in overprivileged access, noncompliance, or excessive consumption of resources. This process means applications ship significantly slower, frustrating developers.

Challenges Following Best Practices

Enforcing consistent configurations across multiple environments is difficult, particularly if there are deployments dispersed across cloud and on-premises environments. Adding to the problem, application coders aren’t always well-versed in security and design best practices, while the operations teams and site reliability engineers may make changes directly to a cloud environment to ensure high availability or security. And although IaC tools provide templates to standardize application and service deployments across environments, configuration drift is still possible. This means that it is critical to detect, track and remediate those configuration changes for every environment. Somehow, teams need to compare the IaC files that were created for the deployment with the ones that exist in the live environment.

Pressure to Deliver Applications Quickly and Securely

Developers are continuously pushed to deliver applications and updates faster. Some teams release updates weekly, while others release code even more frequently. But that’s incredibly difficult to do without context around broader application, infrastructure and business requirements. Platform teams are tasked with doing IaC reviews without the information they need to make smart decisions, resulting in delays and unavoidable human error.

And while platform engineers and DevOps teams are creating golden templates designed to help developers provision infrastructure in alignment with best practices, the developer must still apply the templates consistently and understand how to create secure, consistent and compliant IaC. That is a lot to ask of developers, who just want to build a great application that works the way they designed it.

The Birth of Generative Infrastructure from Code

Infrastructure from code is a new way of thinking about and working with infrastructure provisioning, one that puts the application code at the heart of everything. Rather than creating configuration for infrastructure using version-controlled golden templates, you generate the infrastructure your application needs based on the version of your application being deployed.

Instead of making your application fit your infrastructure and treating the infrastructure like software, making developers learn new languages and infrastructure concepts, you fit the infrastructure to the changing needs of the application, while applying necessary standards and keeping the benefits of version control. This approach significantly improves the developer experience, helping them streamline their processes and accelerate delivery, no lunch-and-learns or workshop training required, so developers can focus on developing rather than provisioning.

Aligning IfC with Changing Needs

As artificial intelligence (AI) and machine learning increase developer output and change how applications are built and maintained, the industry needs a solution that can scale security, automation and efficiency with the output of developer teams. As cloud services release new offerings and countries adopt more cybersecurity and data privacy standards, organizations must ensure that the infrastructure they build is optimized, secure and will not hold up innovation. Even the best templates go out of date quickly and must be updated, and there’s no guarantee that every team will use the updated versions.

By generating Infrastructure from Code, you apply golden standards (rules, guidelines, and best practices that apply to any infrastructure) to the generation of IaC, which ensures alignment with the best practices, configurations, software installations and settings that are approved by the organization. The output is then standards-based and error-free, allowing development teams to truly self-service resource provisioning without worrying that they are putting the company at risk due to a misconfiguration or security vulnerability, and it skips the “scan, report, revise” cycle offered by most policy checks today.

The Cycle of Complexity

We live in an exciting age of innovation when it comes to infrastructure provisioning and management. We’ve seen many new tools that have changed the face of infrastructure and helped reduce the complexity of deploying applications. Kubernetes, containers and microservices all make it easier to develop and deploy software, increasing agility, resilience and scalability. These technologies also require expertise to manage and use them effectively.

While IaC is, undeniably, an improvement on manually configuring hundreds of cloud and on-prem environments, or running custom scripts that are prone to breaking, it has added burdens on developers to learn complex infrastructure architectures and concepts, and to pick up new languages as well as change the way they think about programming. Managing, maintaining and deploying applications and infrastructure securely and consistently remains an incredibly complicated challenge.

Isn’t it time for a new approach, one that puts the application code first?