Tuesday, August 27, 2013

Use Vagrant for a Great Good

Vagrant is the one half of the best tool for IT organizations in the past decade. Hands down. And I'm going to tell you exactly why you are going to believe me.

No-one focuses on it and no-one cares about it, but environment mismatches are one of the biggest problems IT organizations face. It's a silent threat that doesn't take down whole sites. It's more insidious., only biting you every few months. Things that pass QA sometimes mostly work in production. It's really hard to replicate that bug in production. So, you write it off as a heisenbug. Or maybe the test suite passes on the developer's machine and the QA's machine, but sometimes fails in Jenkins. So, you disable that test from running in Jenkins because you've already wasted three days trying to figure it out.

Everyone kinda knows what the root problem is - you bitch about it over lunch every so often. But, it seems like such a high-class problem, so you don't fix it. Yeah, sure, Github and Etsy do it, but those are huge teams with tons of operations resources to put towards making everything perfect, right?

Not really. Both of them are actually small teams, relatively speaking. And, they don't devote huge amounts of time to it. They just do things right from the get-go. There's a bunch of tools these and similar teams use. The first and most foundational tool is Vagrant.

Vagrant is virtualization made easy. Vagrant creates and manages a semi-anonymous virtual machine (VM) using a simple configuration file (called a Vagrantfile). There are three basic commands:

  • vagrant up
  • vagrant halt
  • vagrant ssh
(There's more to it - a total of 15 commands as of this writing, but those are the three big ones.) And they do exactly what they say on the tin - bring the VM up, bring it down, and login to it. It works with Virtualbox, VMWare, and several other virtualization providers.

That's secret sauce #1 - Vagrant is just sugar around virtualization providers. It does all the heavy lifting of setting up the VM, managing it, and making sure it doesn't conflict with other VMs. (Yes, we're going to talk about multi-VM setups!)

So, now you have create a VM. So what? Because the setup of the VM is automated and everything is checked into your source control, every user of this repository has the exact same VM setup on their machine. As the setup of the server changes, a quick vagrant reload and everyone is in sync again.

Setting up multiple VMs is also very simple. You might want to do this for all kinds of reasons.
  1. An application server and its database.
    1. If they're both in the same repository, the same Vagrantfile can define both VMs.
    2. If they're not, each repository has its own Vagrant file. In this case, defining your own subnet works wonders. (I like 33.33.33.xx - it's a private DoD subnet that's not routable.)
    3. Remember - coworkers shouldn't share cups, plates, or databases. It's just not sanitary.
  2. Front-end developers working with services.
    1. The services can run on their own VMs and be deployed to as if they were in the QA environment. Your designers can now work on their code without having to know how the services are managed AND not have conflicts.
So, when do you want to set up a VM? I strongly believe that every source code repository should have its own VM. This includes backend code, like Python or Ruby applications as well as front-end code, like Backbone or Ember applications.

"Rob, really?! Front-end code? Doesn't it run in the browser already? Why go through all the hassle of setting up a VM?"

Yes, really, for several reasons:
  1. Front-end applications may run in the browser, but they aren't built in the browser. Compass/SASS, Less - these are all tools that are versioned and depend on a toolchain of specific versions.
  2. No-one ever works on a single project these days. Each project has its own toolchain, but many of these tools expect to be installed globally.
  3. Most front-end applications depend on some REST API to be available. If it's not, you may choose to build a stub application instead of hard-coding the responses in text files. Now you have a back-end application that needs to be managed.
  4. Test tools often want to run in a server. This is especially true for PhantomJS and ZombieJS. It really sucks when your testing frameworks aren't in sync between developers.
And, finally, Vagrant provides the foundation for the other half of the most important tool of the past decade - the provisioner.