A new Habitat for applications? Chef launches open source app automation project

Chef, IT automation for DevOps, has announced Habitat, an open source project for application automation.

Developing and deploying software is too hard, according to Chef co-founder and CTO Adam Jacob. An application may work fine on a developer's laptop or as a prototype on a single VM (virtual machine).

"However, if you want to run a production environment across thousands of servers, it gets significantly harder. Most of what's hard about it is about, how do I build the application that goes in the virtual machine? How do I build the virtual machine image? How do I orchestrate the way to deploy? How do I know that deployment is safe? How do I handle dynamic topology? How do I do service discovery, if my application depends on 15 other services for it to run? How do you know where they are? How do you update them? Those are the sorts of problems that are intractable," he told The Register.

His solution is Habitat, which went public today under the Apache license.

What is Habitat? The core concept is that instead of deploying applications as-is, you deploy them as packages that are managed by a Habitat "supervisor." A Habitat package includes the software and all its dependencies and configuration options. The supervisor manages the lifecycle of the package for which it is responsible, and is able to start, stop and update it. Habitat supervisors can be linked to form "rings," which are groups of services. Habitat is agnostic about where it runs, which could be in a container or on a VM or on a physical server.

The Habitat command line tool lets you build and install packages. It is most at home on Linux, but also works on Mac OS X in conjunction with Docker Toolbox, which creates a Docker environment on a Mac. A Windows version is on the way.

"We're going to work closely with Microsoft to get the Windows port of Habitat so that you can write Windows native apps and run them under the Habitat supervisor," said Jacob.

The software in a package is ideally built from source, but can also be a proprietary binary. Habitat records all the dependencies and version information in a manifest. You also include configuration files in a language called TOML (it stands for Tom's Obvious, Minimal Language).

Using Habitat, "all the behavior of the application through its entire lifecycle lives with the application itself as it moves through the lifecycle. It doesn't need any other system in order to behave correctly. So if you deploy the application and it's using Habitat, and you put it on bare metal, you manage it the same way you would manage it if you deployed that application on Cloud Foundry, or if you deployed it on Docker, or Mesos, or Kubernetes," said Jacob.

The capabilities of Habitat supervisors, when linked, include dynamic configuration of services, called topologies. "An example would be, you have a database. And you need to have one leader and lots of followers, so you're replicating all the data that comes into the database. The supervisor can manage that for you without you having to do anything. You don't have to deploy a configuration management server, or service discovery, or a deployment orchestration engine, or any of those things. You just tell the supervisor: this service runs in this topology, and it just works," Jacob told The Reg. "As you deploy the application, you are also deploying the automation."

Is Habitat middleware? "We describe it as application automation because we think that is the best possible fit for what it is. In a way it's middleware. It provides a bunch of services that an application can use. What's very different is that traditionally, you write the application for the middleware. That's backwards from Habitat. Habitat wraps around your application as it is. It's actually a new way of solving the problem," said Jacob.

Habitat itself is written mostly in Rust, a systems programming language created by Mozilla. "It makes some really great guarantees about safety. And it has no runtime, so it allows the supervisor itself to be very small. We love Rust," Jacob said.

The most telling part of my conversation with Jacob came toward the end, when I asked what benefit Habitat brings to those who have already embraced fully automated application delivery.

"If you've gone through the trouble to understand how to do it the hard way, Habitat is like doing that in easy mode," he said. It sounds like a confession that doing DevOps right is currently too difficult.

The project has initial support from others including Intel, Mesosphere and CoreOS. Future plans include an automated public build service as well as extending Habitat to cover more scenarios and platforms. An intriguing project, which you can check out here. Information about ChefConf 2016 in July can be found here. ®