Continuous Integration is a fancy term for “run your project’s tests after someone pushes to the repository and notify interested parties if they fail.”
We’re currently in the process of revamping our test suite (which we’ll blog about in the future) and moving servers, so I thought it’d be a good time to re-evaluate our options.
Integrity has grown a lot since its first release. It has a ton of features, great documentation, and nice notifiers (I wrote the Campfire notifier).
It also has a very attractive interface, is easy to configure, and works with multiple projects. And it’ll run anything – not just Ruby projects.
It’s not the easiest thing to install, though. There are a lot of dependencies and I never quite got it working on my latest install attempt. I hear it works better with Passenger than Thin (what I was using).
If you can get it working, it’s worth it. qrush’s report card is a great addition, too.
BuildBot is a Python builder that’s pretty feature complete. And, of course, it works great with GitHub.
I installed it and tried it out – it’s pretty easy to use. And because it’s a generic builder you can also use it for non-test related tasks, like compiling stuff. It has a server BuildBot and worker BuildBots which means you can scale it to run many concurrent tasks, even across machines.
For us it seemed overpowered, but I’ll definitely keep it in mind if we need hardcore lifting in the future.
RunCodeRun is Relevance’s hosted CI service. It supports both private and open source projects, but is Ruby-specific.
For an example check out the AASM project overview or a specific build view. Pretty nice. It has tight GitHub integration, too, linking to each commit.
Unfortunately RCR doesn’t support Campfire notifications yet, as far as I can tell. We need ’em!
CI Joe
Because knowing is half the battle. CI Joe is a dead simple, Git-talkin’, Unix-lovin’, HTTP slingin’ continuous integration server we wrote to do one thing and do it well.
It uses your Git config and lets you extend it through Git hooks. A POST will trigger a build – which means it works great with GitHub. It supports HTTP auth so Internet prankster can’t trigger your builds. It comes with Campfire support. It’s language agnostic – as long as your test suite can be run from a Unix shell, CI Joe can handle it.
Check out the documentation for the complete tour.
We use the Campfire notifier (I sound like a broken record, don’t I) and Joe’s HTTP basic auth feature. Our config looks like this:
$ cat .git/config ... [campfire] user = notifier"github":http://github.com/github.com pass = secret subdomain = github room = The GitHub Dancy Party [cijoe] user = chris pass = secret runner = rake -s test:units
We also use Joe’s “after-reset” hook. We keep our database.yml file in Git, but the CI server needs its own database config settings. If Joe finds an executable “after-reset” Git hook it’ll run it after updating the repo and before running the tests. Ours looks like this:
$ cat .git/hooks/after-reset rm config/database.yml cp database.yml config/database.yml
As you can see, we keep our good database.yml unversioned in the CI clone’s root and just remove the versioned one after each reset. Joe runs a “git reset —hard” which does not remove unversioned files – our custom database.yml won’t get wiped.
Again, check out the source and documentation to get rollin’. Issues go to Issues or the mailing list.
Enjoy.
We have been having some nice success with hudson https://hudson.dev.java.net/. Has some nice features and a large amount of plugins. I would recommend checking it out too.
pallan: Thanks for sharing!
If anyone else has CI suggestions, feel free to leave them in the comments.
Cheap interns. Just kidding.
Thanks for the article. I've been thinking of trying a few new setups. This will definitely get me started.
I'm a huge buildbot fan -- to the point where I do use it where it's overkill. It's completely ideal when you want to support a large number of different platforms with different branches and different build configs. I did a bunch of work on the git support for "try" builds in such a way that understands distributed revision control better.
From within my local memcached directory I can type
buildbot tryand the code I have locally (both locally committed and even uncommitted) will build and test against our thirteen builders.I don't even have access to many of the builders, so it fits the distributed model quite well.
Of course, it also automatically builds everywhere after a push to github and all of the realtime views light up.
I'll second the nod to hudson. It's one of the best Java apps I've seen.
We use cerberus for doing CI tests on about 10 rails apps. http://github.com/cpjolicoeur/cerberus/tree/master
Works like a dream. Really easy to setup. Supports multiple branches and git/svn.
I'll add another +1 for Hudson. It is really a mature solution and out of all of the open source ones I've tried it is by far the best.
Hudson is a great continuous integration server for rails/ruby apps.
Pretty easy to setup and run, built in git support and supports a bunch of plugins. One feature that has saved our ass many times is the builders stop if the hard disks get full, or if any of the vitals fall outside their range.
I'm also a very pleased user of hudson.
We looked into Hudson but wound up rejecting it partially because it did not have a Campfire plugin. Campfire as become an integral tool in our dev environment, and we do not have the resources to write our own plugin.
We are currently using Cruise Control, but are starting to look at other solutions.
CruiseControl! :)
I'm a long time contributor to CC, now working at Urbancode, makers of AnthillPro. AnthillPro doesn't have a Campfire plugin yet... but I've been looking for a good project to play with our plugin API.
How about gitbuilder? http://github.com/apenwarr/gitbuilder
How about CMake/CTest/CDash ?
Btw, if you want to get some informed opinions about CI tools checkout the mailing list for CITCON, the Continuous Integration and Testing Conference: http://www.citconf.com/contact.php
CDash public dashboard: http://www.cdash.org/CDash/
The full pack: http://www.cdash.org/cdash/project/about.html
(make, document, test, installer, CI etc.)
What about local hooks? I have found that local git hooks to be one of the more powerfull features of Git and distributed revision control. Arora (http://github.com/Arora/arora/tree/master) includes a number of local git hooks that the developers run to make sure that what they are push is already in a halfway decent and sane manor. Arora just about never has a commit pushed that breaks the build or tests. Because you don't want to force push a main repository one that can only be run locally is spell checking on git commits messages. This simply and trivial git hook lets the user do a quick git commit --amend before they make the mistake public.
No Campfire plugin for Hudson? Did you look at this one? http://wiki.hudson-ci.org/display/HUDSON/Campfire+Plugin
If it doesn't meet your needs, feel free to make a feature request.
I just checked out hudson. It's very cool.
On closer inspection, the plugin described on that page seems to be vaporware...
I just took a quick look at integrity-campfire and it would be pretty trivial to turn that into a Hudson plugin, assuming that you could run the Tinder code under JRuby :) Until JRuby supports Java annotations it would be cumbersome to implement a Hudson plugin in JRuby, but my understanding is that that support is not far off. For the time being, the plugin would most easily be implemented with a bit of Java, but could ideally use the same Tinder code via JRuby.
Looking forward to the test suite post ... I'm still doe-eyed and my hardened industry friends laugh when I mention "automated testing".
we use hudson and i should say its pretty good
see our huson working at http://build.openbravo.com
What's is the relevance of CIJoe when there's so many other CI servers that seem like they would fit your needs? CIJoe looks like a single-project, github-oriented version of Integrity, but Integrity still supports the features that you needed like GitHub post-update hooks, Campfire notifications (and a lot of different other ones, Email, IRC, Jabber, Twitter), etc etc.
I have had the same problem with integrity. It always takes me a lot of hacking around to make it work. I thought the newer version would be better, but I just tried a fresh install and out of the box on passenger. No CSS or Images. Followed by can't insert the data in the DB as the table doesn't exist, even though it was just freshly migrated...
Not sure why there is such love for something that I can barely ever get to work... Note we still have a hacked version running as our CI, but it is flakey and I am afraid to mess with anything for fear of breaking it...
OK after my rant spent a bit more time debugging and this time it was my fault... I still had an old version of foca-integrity installed on my system and the two were fighting with each other... So trying again completely fresh with integrity on passenger for the first time worked out of the box for me, so I have to give credit where due, and say I didn't have the issues the GH team did getting it set up.
Git hooks are great and all, but what if your server is down when the hook fires? Yo miss a commit. You don't build. You don't know that commit you missed just broke everything. Nobody checks in for weeks, and your project is broken for weeks. You look silly. You lose the battle.
Any serious CI server must have log polling support.
-- Chad
Here are a few recent RunCodeRun updates that may play a role in helping you choose a CI solution. ;-)
More to come. Stay tuned.
Now Hudson has a Campfire plugin: just released a simple plugin to provide build notifications to Campfire rooms:
see http://wiki.hudson-ci.org/display/HUDSON/Campfire+Plugin
or http://schneide.wordpress.com/2009/10/26/a-campfire-plugin-for-hudson/
for more