Simplify testing of PRs

[alxy]

In this PR I want to discuss ways to simplify the testing and itnegration of new features provided by PRs. My feeling is, that currently there are a lot of PRs that just need appropriate testing to get merged. Examples include #4700, #4652, and many more.

However, doing the testing is not so easy (I tried it myself), as you definetly need a git based installation to pull in the required changes. Unfortunatly, most people do not even know how to set up october to start testing, as they either use the wizard or composer to create a new october project. There is no guide that I know of how to get a working installation of october using the git repository (it is definetly not too difficult, but still may be challenging when doing it the first time).

Proposal
Provide a simple guide how to set october up for PR testing and how to checkout PRs locally. It would be even nicer if we could have a Docker container with October and the test plugin installed. Maybe even with a small helper command to directly checkout certain PRs by ID - not sure if this is possible as I havent used Docker yet.

What do you think?

[bennothommo]

@alxy We do have a step-by-step in the .github/CONTRIBUTING.md guide, but given it's mainly Git-based, I'm definitely on board with trying to find ways of making testing even simpler.

One idea that popped into my mind just now is perhaps having our GitHub Actions bot post a comment whenever we add the "Testing Needed" label to a task, that can provide a one-line command to run Aspen Digital's fantastic OctoberCMS Docker image, as it can be configured to automatically pull in the code in a pull request on boot. Whilst Docker is still a semi-advanced tool, it might be more acceptable for people to just run the command and have a working copy of October CMS (with additional pull request changes) immediately ready for testing.

[LukeTowers]

@bennothommo I just had an even better idea. @petehalverson what do you think about making it so that we can trigger octodock builds from the GitHub actions bot? The idea would be that a new build could be triggered from a commit being added to a PR (maybe with a slight waiting period to allow for multiple commits to come in at once) which would then post the link to the container as a comment on the PR and automatically destroy / rebuild the container when new commits are added.

[petehalverson]

Yes! I haven't had a chance to play with GitHub actions much, but had plans to do the same in OctoDock via webhooks.

[LukeTowers]

@petehalverson as long as you can provide a URL for us to hit, I think GitHub actions can trigger it. Although if webhooks are easier for you to work with I'd be fine adding them to the repos as well.

[petehalverson]

I'd like to play with GitHub Actions a bit more to see what's possible. My understanding is that it can be used to build and push images but I don't think we could use it to host a running container for testing.

Regardless, I think it'd be a good idea to get a webhook setup to trigger builds on octodock. I'll follow up when I've got that.

[LukeTowers]

@bennothommo I do like the idea of having the bot create a one-liner for people to run locally though in the meantime, is that something you're able to setup?

[bennothommo]

@LukeTowers sure thing.

I did consider your idea of having the builds automatically running in Octodock (if @petehalverson did make it available), but I thought it might not work for two reasons:
a) As Pete said, GitHub Actions cannot make automated Docker images web-accessible (and would destroy them after an hour anyway), and,
b) People may wreck havoc on publicly accessible instances.

[petehalverson]

I envision it as a link the GitHub Actions bot adds to the PR comment thread. The link would start a fresh container on OctoDock with the latest develop image and PR.

The entrypoint script already supports merging PRs at runtime, it's just a matter of following up on some tighter integration with the repo and introducing some rules to avoid abuse to host them publicly on OctoDock.

[petehalverson]

@LukeTowers @bennothommo here's an example one-liner for local use:

docker run --rm -p 80:80 -e GIT_MERGE_PR=4758 aspendigital/octobercms:develop

[github-actions]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.

[github-actions]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days.

• If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue. - If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[bennothommo]

Marking this as on hold for now. I have attempted to do this via GitHub Actions, however, GitHub Actions does not allow you to do any "write" operations in an Action that is run by a PR from another repo - ie. nearly all of our PRs. This is the same reason that our "wrong branch" action does not work for external contributor PRs.

I will attempt to find another way to handle this.

[bennothommo]

We have deployed a command that we can run in GitHub to present the one-liner Docker instance as @petehalverson suggested - you can see an example here: #4965 (comment).

Ultimately, while this will provide a decent shortcut to getting October ready for testing, the only thing that will truly speed up the testing of PRs is if the community gets testing with us as well :).