Require passing CI build before building a recipe?

[DarwinAwardWinner]

I've just set up Travis CI for my ido-completing-read+ package (here), and I was thinking that it would be nice if MELPA could check the CI build status for the master branch and avoid updating the package if the CI status is not passing. Obviously this would not be something to enable for all packages, so I'm imagining adding an option to a package recipe to tell MELPA where to find the CI status for that package. Do you think this is feasible? I know that for my package, if any of the tests fail, I definitely want to prevent that version from getting to MELPA until I fix it.

[DarwinAwardWinner]

I should say that I'm willing to write some code to implement this. I just want to know if it's something you would be interested in first.

[purcell]

We don't really have any mechanism for nicely handling failure of builds, to be honest, so this is potentially a can of worms from a purely technical point of view. And we don't have any mechanism for authors to determine that their package has not built, and why.

But in any case, I would prefer if MELPA simply contains a package of the latest master code, whatever its state, even if that very occasionally means "broken", for some value of that term. MELPA Stable allows for packages that have been verified in some way.

[DarwinAwardWinner]

You mean MELPA doesn't have a way to reject the latest master and keep using whatever version it had previously?

[purcell]

Well, if the build of a package fails, the old version isn't replaced. But the build failures aren't surfaced in any helpful way, and the only failures we currently expect are catastrophic ones, e.g. upstream repo. disappearing, or malformatted recipe / package descriptor file. There's otherwise no concept of verifying that a package is "good".

[DarwinAwardWinner]

Ok, fair enough. I guess this would require significant changes to the inner workings of MELPA's build process, then.

[purcell]

In the longer term I'd like to run package-lint across packages both as a gate for Travis (to make initial review of new packages more efficient) and as a source of more details to show on the package's MELPA web page. Idea is to make it clearer which packages are more compliant and well-maintained. I can see upstream build status potentially being a factor then.

[DarwinAwardWinner]

Ok, sounds good to me.