Multiple CI gates

[alexcrichton]

This may already be implemented, so it could just be a clarification issue, but if we enable both buildbot and Travis for the main Rust repo, will homu wait for both to finish? e.g. will a success in one and a failure in the other cause the whole build to fail? Also, will merging hold off until both have succeeded?

[barosl]

No, currently Homu only chooses one configuration per repository. This must be something I overlooked, I just thought either Buildbot or Travis is enough. Is this feature needed by the Rust project?

[alexcrichton]

Ah ok, in that case I've tweaked the title to be a feature request. We're thinking of starting to gate on Travis (in addition to buildbot), so it'd certainly be a nice-to-have! Not super critical though.

[indirect]

The Bundler team would like to start gating on both Travis CI and code style/lint checker (reported as a separate GitHub commit status), so this would be super helpful for us.

[larsbergstrom]

For our non-Buildbot configs, Servo would like to test our PRs on TravisCI for Linux+OSX+Android and AppVeyor (#87) for Windows, so this would help us, too.

[larsbergstrom]

Implemented by @Manishearth in servo#13

[Manishearth]

Note that Rust is very far behind on homu versions (last I checked), so if you want to use the above be very careful when upgrading; a lot has changed and broken. I recommend backing up the homu dir (with db and venv), and perhaps starting off with a fresh homu install.

(With Servo I usually move the homu dir to a homu_back, and re-highstate salt)

[Manishearth]

Also needs servo#14 to work on repos not using status

[cgwalters]

I'm thinking the architecture might be saner if rather than having Homu support multiple builders directly, anyone who wanted that would have a small shim project that implemented the subset of the BuildBot API we're using. So from Homu's perspective there's only one build - and all of the questions @alexcrichton asked in the start can become the choice of the Rust project.

In practice perhaps we include in the tree a shim API to do Travis + BuildBot, or in my case I'd like Travis + Jenkins in the source tree.

(I haven't looked at this in detail, just armchair architecting...)

[Manishearth]

That sounds needlessly complicated. We already have a standard API, and that's the github status API which everyone knows how to use (except buildbot, but we already have special casing for that -- and with some effort we could make buildbot feed to github statuses per-builder if that ability doesn't exist in a plugin already). This makes it ridiculously easy to add new builders. It's also easy to write shim hooks that can OR statuses to create new statuses, which homu can consume, if necessary.

The questions Alex asked were about enabling Travis for Rust, with the assumption that enabling Travis will change how homu behaves (it won't, it will only change if the homu config is changed). I'm not sure if anyone actually wants the ability to OR statuses. However, like I said, it's easier to do that than to shim around the buildbot api.

[cgwalters]

Ah, true, using the github status API as the central state point would make sense indeed.

One thing I'd note though: going that way I'd say reinforces #119 (comment)