Ubuntu Syndrome.

Feb 11, 2015

Warning: Bitter BOFH ahead.

There is a phenomenon I've come to call Ubuntu Syndrome, after the distribution of Linux which has become the darling of nearly every hosting provider out there (and no, I won't call them bloody cloud providers). All things considered, it seems to have a good balance of stable software, ease of use, availability, and diversity of available software. It also lends itself readily to the following workflow:

  • Use a tool like packer.io to automagically instantiate a copy of Ubuntu at the hosting provider of choice.
  • Never patch the machine under any circumstances.
  • Use Chef, Ansible or another devops* tool to install Ruby On Rails or node.js.
  • Install Git with said devops tool.
  • Check the latest commit of the application in question out of the project's Github repository and run whatever build process is necessary (because yes, today we have to compile freaking scripting languages) to set up the application.
  • Start the application (thankfully, no longer as the superuser by default).
  • Don't set up any system level monitoring of any kind. Only make sure the application stays up.
  • Find out your production VM has been pwned weeks or months later.
  • Terminate the VM. Archiving the disk image to perform a forensic analysis shortly before the heat death of baryonic matter in the universe is entirely optional.
  • Start over from step zero.

Look. I get that virtual machines are, for all intents and purposes disposable. They're cheap to stand up, relatively cheap to operate (up to a point), and trivial to tear down so you can start over. They're certainly more convenient than having to rebuild and reinstall an entire physical server from scratch. On the other hand, there is a lot to be said for doing things right up front so that you can skip over (or at least hopefully postpone) the whole "get pwned" part of the show. A little bit of extra work up front (like running the command apt-get update && apt-get upgrade -y) can save a great deal of time and effort later by installing the latest and greatest security patches. It takes a little while, sure, but why work extra late nights if you don't have to? In addition, there is something to be said for hardening your VMs when you stand them up at the same time you patch them to make it that much harder for the VM to be compromised. It doesn't take long; in fact it can be as simple as copying a handful of files and rebooting the VM. Here's my private stash of hardened configs for Ubuntu v12.04 and v14.04 LTS that I deploy on all of my servers (virtual and otherwise, when I have to use Ubuntu). There are other resources out there, sure, but these are mine and you're welcome to use them.

Put a little thought into it. Just because something is disposable doesn't necessarily mean that it's worth extra trouble and hassle later. Save yourselves the energy for more interesting things later. * Ignore all of the fancy pants definitions of "devops" you'll find on the Net, they're all bollocks. DevOps is the trendy word for "automating the shit out of everything that can be automated."