Make contributions easier by making PR requests testable

[fiasco]

When a PR is contributed to govCMS there are several loops it has to get through to get tested. The tester has to be able to:

• Clone down the PR and check the branch out in git
• Compile govCMS
• Install govCMS
• Test the change

For someone who is not technical, there are a lot of technical steps to make this happen. If we could automatically setup the PR in a testable environment then this would help us get more community members involved by being able to test changes without having requiring the technical skills necessary to build and host a govCMS site.

[stooit]

This work is largely done. I've built an example on my govCMS fork to illustrate how this will work:
https://github.com/srowlands/govCMS/pulls (you can submit PRs against this fork to test, too)

There are two example PRs:

• Make the background yellow
• A failing Behat test with screenshot << check the screenshot in comments

To check out the environments and failure screenshot you'll need to add /etc/hosts records (the 'real' version should have a wildcard DNS record) :

• 52.64.233.163 3.stage.dev.govcms.gov.au 4.stage.dev.govcms.gov.au

Any submitted PRs will trigger a new environment to be built, whereby the govCMS codebase is built and tested against a merged codebase. The environments will persist, so a user will be able to login and test features or enhancements made in a pull request.

If one of the tests fail, a screenshot will be taken (via a headless webkit browser) and be made accessible, making debugging issues easier.

There are a few things that we need to build the 'final' version of this:

• how should we handle authentication to the PR environment? (Perhaps a default admin user/pass shared by request?)
• which domain should this run off? My test env assumes 'dev.govcms.gov.au'. A PR environment may end up being $ISSUE_ID.dev.govcms.gov.au?
• how should we handle DNS + SSL? A wildcard SSL certificate would be nice to ensure we are testing in an environment that closely mirrors real govCMS
• a 'bot' user should post the results of a build (via the comments). I'll need a token to be configured on an appropriate user (to be discussed)
• we can also do improve the generated comments (like pasting in the failure messages from the console)

[fiasco]

Probably unlikely that we'll be able to publish PR environments under a .gov.au domain - even if it does have "dev" in the name. Perhaps we buy govc.ms to run these does off? E.g. $ISSUE_ID.dev.govc.ms? This would probably also make it easier to obtain a wildcard SSL Certificate.

For the bot, can we create a user? "govCMS Jenkins" or perhaps Jim?

A nicely formated pass/fail post would be sufficient with a link to the build failure or site build.

[stooit]

Yes we can create a regular GitHub user to be the bot - I'll need access to the user account to generate an appropriate token.

We have a wildcard domain hooked up to the test environment now (thanks @typhonius), so there is no need for /etc/hosts records, and we can now post screenshots directly into the response comment from the bot (see https://github.com/srowlands/govCMS/pull/4)

[fiasco]

govCMS bot is up so I think we can close this one. @SRowlands++