Community CI service

[hueniverse]

The hapi ecosystem is getting big, much bigger than anyone can keep track of. This is particularity true for hapi itself. It is becoming very hard to ensure that changes which pass all the hapi tests, are not breaking other modules, including tightly coupled ones like inert (file and directory handlers), h2o2 (proxy handlers), and vision (views engine).

It would be awesome is someone(s) will take on the project of building a community service where people can register their open source work, and every time there is an update (publish or commit, open for debate) on any of the core framework modules, it will test those out.

Some of these projects can be nothing but tests to ensure your private code will not break without exposing it.

The npm package popular is still available...

[marcus7777]

sounds like a great idea where do we get started?

[hueniverse]

My guess is that we need a service somewhere that gets hooks from github on changes (or npm on publish) and triggers a travis build of everything and then published the status on a page. The service can live on hapijs.com and the code should live in a repo under the hapijs org.

If you want to give it a shot, start as your own project and see how far it goes.

[yonjah]

👍 sounds like an awesome idea. I guess the actual CI is not the hardest part, but the registration of packages and tests. @marcus7777 send me a msg if you need help.

[timmarinin]

I'll try on weekend :)

[yonjah]

Which CI server were you thinking of using ?
Travis looks good, but we might need to create some templates for .travis.yml file which will need to be placed in the tested repository (I don't think there is any other way to change the build process or even control which repo branch will be tested without this file)

[marcus7777]

I'd love to help with this but i'm not quite sure how i've not worked with Travis but i've been reading up on it for a week sorry for not getting back to you all. Thanks @marinintim maybe i can help when you get it started.

[timmarinin]

Oh, I was busy at this weekend and didn't touched anything yet.

@yonjah, I was thinking about Travis. Need to dig more, yeah.
@marcus7777, that would be cool 😎

[yonjah]

@marinintim Haven't had the time to code anything yet just tried to get my head around how it should work.
@hueniverse is there any preferences to the DB it should use ?

[hueniverse]

@yonjah I would prefer something hosted and simple.

[timmarinin]

Need clarification for the flow:
When new commit is pushed to repo, then everything that uses this repo goes to test.

It's like reverse deps.

[Marsup]

It won't work if you trigger your builds on commits, new builds should happen only when a new release is pushed to npm, otherwise tests wouldn't take the latest version.

[timmarinin]

@Marsup, correct point, thanks.

[yonjah]

@Marsup I thought thats the hole point to have the testes running before releasing a new version.

Did anyone got to work on it this weekend ?
I think I'll start now to see if I can get something going with Travis if any one want to help, or maybe take the registration side of things that would be great

[Marsup]

How do you plan on making that work ? Travis installs through npm, if nothing is published I don't see how you get it tested.

[yonjah]

You can set a github repo for npm dependency so commits shouldn't be a problem.
You can also run custom commands after / before install with the .travis.yml file so I thought of using a yml file that will change the repo according to a specific environment variable ( either changing package.json pre install or replacing hapi folder after install ).
I'm still early in the stages of working with Travis api but I think this should be possible

[Marsup]

Let's say hapi has a branch hapi8 that depends on inert which also has its own hapi8.x branch which depends on boom that has yet another hapi8 branch.
Considering you probably can't rely on branches names, what makes you think you can get the dependency tree right ?
Kudos for trying but I think you'll only get false positives if you're integrating on commits.

[yonjah]

But I thought that was the point. To always rely on the latest version.

If you need to rely on branch names you can do it, but I think conventionally the master branch should contain the latest version. If you look at Hapi core repos they are not using any branches at all (only tags) but even if they do depending on a specific branch shouldn't be an issue.
So if the core apps we're defining are hapi, inert, and boom. Any dependency of them in the tested repo will be replaced with the latest committed version, and on any commit the them the test will run.

I guess we can limit the testing after publishing, but then you might only find errors after publishing ( and I guess if many external libs fail you might prefer to fix the changes to be more backward compatible).

Any way after going over Travis API today I'm not sure what I thought of doing is possible.

1. you cannot run tests and builds for repos you don't own (even public ones) so you need to keep a github token for each user that register his repo
2. it seems like there is no way to run a new build without a commit (you can only restart an old build)
3. you cannot set variables for a specific build only in general for the repo.

So this basically mean that any repo that is added to the system has to be specifically crafted to work with the system, either having a specific branch that will run the test or that every test will run with the latest version of hapi.
Originally I thought of allowing users to have their own testing mechanism with a fixed version of hapi and have a way to run specific test with the updated version but I don't think this is currently possible.

So as I can see the system working now, a user will give us a token to allow us to access his Travis system will choose some repos from it he wants to build, and we will just rebuild them whenever we think its necessary. since were not setting any specific variables the user can decide if he wants to run npm update hapi, include hapi master branch as dependency or whatever else they want to do in their travis settings.
The only thing the server will do is to restart the last build

This is kinda annoying but unless we want to use some other CI. It seems like other hosted CI like CircleCI and Semaphore are supporting this feature I'm just not sure how free they are and from what I usually only see Travis being used in open source projects.

[Marsup]

Hapi core repos use branches, most of them have been deleted since the release, but master will probably do.
Ideally travis-ci/travis-ci#249 should deal with that, but in the meantime I'd go for a very simple service acting as a gateway to travis API and set up IFTTT RSS rules on repos I want to watch then forward it to the CI service to make the last master build run again.

[yonjah]

I worked on this a bit over the weekend.
You can see the result here -
http://ci.non.co.il

and the source here
https://github.com/yonjah/hapi-flow-server

It is still in very early stages, but I think its enough to play with.

the main thing missing is ui for adding repos and github hooks.
but the api is there.
so to add you lib to be tested just run -

curl --data 'url=[GITHUB_URL]&branch=[BRANCH]&test=[TEST_CMD]&report=[FILE.XML]' http://ci.non.co.il/repo

this command will return a new repo instance from the db

{
"url":"https://github.com/yonjah/hapi-sol.git",
"name":"hapi-sol",
"type":2,
"branch":"latest",
"test_command":"npm run test-xml",
"report_file":"result.xml",
"jobs":[],
"_id":3,
"hook":"http://ci.non.co.il/hooks/3"
}

the main thing to notice is the hook url.
this url is the one you need to add to your github pull hooks to initiate a new test.

To add hapi core module (like boom, joi etc.. ) add type=core to the request payload.
I already added hapijs/hapi. so all your pushes will be tested with hapi latest
but no testes will run for hapi commits until the following push hook will be added to the repo -
http://ci.non.co.il/hooks/2
it would be great if @hueniverse or somebody else from hapi team do it

let me know if you need any help testing this out

[arb]

Hey @yonjah can you get on irc in #hapi? Let's talk it out to push this thing forward. I'm arb in there.

http://hapijs.com/community for irc information.

[yonjah]

@arb I'm not sure if it will work with our time difference.
I'm there now as Yonjah. but I'm not sure I'll notice any new messeges
maybe its better to have a chat on gitter I created a room

[DavidTPate]

It looks like this could be handled with Travis here shortly based on travis-ci/travis-ci#249

It'd be really nice to use their infrastructure instead of having to maintain a set of infrastructure/code to handle this.