Buildbot

Buildbot is a flexible, opensource, continuous integration server. It is entirely free (unlike TeamCity, TravicCI, ...) and it is trusted by some pretty major projects: (python,twisted,mozilla,google-chromium,...).

It takes a little more setup than the others, but in my opinion, the additional setup pays for itself.

NOTE:

Okay, so here is the grand plan. Vagrant is fantastic for creating a lightweight base-image of an OS to get you up/running. The disadvantage of vagrant, (to a testing environment) is that it's changes are permanent. Docker is better for testing (changes are committed, you are only playing in short-lived droplets, copies of a source-box).

So we have 2x options:

fully setup vagrant base-boxes
Wait for system to boot, run tests, throwaway the entire vm.

using vagrant to host docker instances
Use a generic Vagrant box (one per operating system). Inside that VM, use Docker to recreate your dev environment. Run tests in quick spinoff docker instances of that image. (would be much faster, and a single unix machine could run it all)

• use pre-created vagrant base-boxes to host native docker instances. Docker instances would be tailored to our dev needs.