Create a slash PR bot to run our ./build.sh command

[echoix]

Fixes #2351

Proposed Changes

1. Add a slash command dispatcher.
2. Add a help command
3. Add a build command

Readiness Checklist

Author/Contributor

• Add entry to the CHANGELOG listing the change and linking to the corresponding issue (if appropriate)
• If documentation is needed for this change, has that been included in this pull request

Reviewing Maintainer

• Label as breaking if this is a large fundamental change
• Label as either automation, bug, documentation, enhancement, infrastructure, or performance

[echoix]

I’ll add the change log later this afternoon, my time. I’m running late

The checkov didn’t like the workflow_dispatch input arguments.

[echoix]

I could’ve force pushed to clean everything, (the only way to test was to make pushes).

What’s fun with using workflow_dispatch type is that we can even test changes to a command that is in the PR (once the dispatcher and an empty one with the same name exists in the default branch). That way, I didn’t have to commit to main each time I wanted to test.

The commands will only work for people with write access to the repo (it can be one of the 5 levels of permissions). It is configured in the slash command dispatcher.

[echoix]

More commands can be added by adding the name to the slash command dispatcher, and having a new workflow file called the same, but with -command.yml at the end

[echoix]

With cache on Python dependencies, running the /build command runs in about 1 min 30 s.

The eyes reaction is added when the command dispatcher processed the command comment. The rocket reaction is added when the command is launched, and the hurray reaction is added when command is completed (inside the command)

[echoix]

Oh, and the comment is edited during the run to show progress :)

[echoix]

🦙 MegaLinter status: ⚠️ WARNING

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ BASH	bash-exec	6		0	0.02s
✅ BASH	shellcheck	6		0	0.13s
✅ BASH	shfmt	6	0	0	0.04s
✅ COPYPASTE	jscpd	yes		no	2.88s
✅ DOCKERFILE	hadolint	114		0	13.47s
✅ JSON	eslint-plugin-jsonc	21	0	0	1.81s
✅ JSON	jsonlint	19		0	0.22s
✅ JSON	npm-package-json-lint	yes		no	0.68s
✅ JSON	v8r	21		0	11.92s
⚠️ MARKDOWN	markdownlint	309	2	230	5.45s
✅ MARKDOWN	markdown-link-check	309		0	5.48s
✅ MARKDOWN	markdown-table-formatter	309	2	0	15.0s
✅ OPENAPI	spectral	1		0	1.7s
⚠️ PYTHON	bandit	183		47	2.05s
✅ PYTHON	black	183	0	0	3.53s
✅ PYTHON	flake8	183		0	1.8s
✅ PYTHON	isort	183	0	0	0.42s
✅ PYTHON	mypy	183		0	6.89s
✅ PYTHON	pylint	183		0	10.94s
⚠️ PYTHON	pyright	183		244	17.47s
✅ REPOSITORY	checkov	yes		no	28.18s
⚠️ REPOSITORY	devskim	yes		61	1.29s
✅ REPOSITORY	dustilock	yes		no	2.45s
✅ REPOSITORY	git_diff	yes		no	0.05s
✅ REPOSITORY	secretlint	yes		no	7.88s
✅ REPOSITORY	syft	yes		no	0.88s
✅ REPOSITORY	trivy	yes		no	21.18s
✅ SPELL	cspell	745		0	16.93s
✅ SPELL	misspell	566	2	0	0.56s
✅ XML	xmllint	3	0	0	0.03s
✅ YAML	prettier	81	2	0	2.34s
✅ YAML	v8r	23		0	58.4s
✅ YAML	yamllint	82		0	1.25s

See detailed report in MegaLinter reports

You could have same capabilities but better runtime performances if you request a new MegaLinter flavor.

• Click here to request the new flavor

MegaLinter is graciously provided by

[echoix]

🦙 MegaLinter status: ⚠️ WARNING

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ BASH	bash-exec	6		0	0.02s
✅ BASH	shellcheck	6		0	0.13s
✅ BASH	shfmt	6	0	0	0.36s
✅ COPYPASTE	jscpd	yes		no	2.75s
✅ DOCKERFILE	hadolint	114		0	16.56s
✅ JSON	eslint-plugin-jsonc	21	0	0	2.11s
✅ JSON	jsonlint	19		0	0.24s
✅ JSON	v8r	21		0	12.21s
⚠️ MARKDOWN	markdownlint	309	0	230	6.59s
✅ MARKDOWN	markdown-link-check	309		0	5.57s
✅ MARKDOWN	markdown-table-formatter	309	0	0	17.62s
✅ OPENAPI	spectral	1		0	1.63s
⚠️ PYTHON	bandit	183		47	1.99s
✅ PYTHON	black	183	0	0	3.62s
✅ PYTHON	flake8	183		0	1.75s
✅ PYTHON	isort	183	0	0	0.73s
✅ PYTHON	mypy	183		0	6.89s
✅ PYTHON	pylint	183		0	11.58s
⚠️ PYTHON	pyright	183		246	17.44s
✅ REPOSITORY	checkov	yes		no	29.15s
✅ REPOSITORY	git_diff	yes		no	0.35s
✅ REPOSITORY	secretlint	yes		no	14.14s
✅ REPOSITORY	trivy	yes		no	27.22s
✅ SPELL	cspell	745		0	19.81s
✅ SPELL	misspell	566	0	0	0.93s
✅ XML	xmllint	3	0	0	0.36s
✅ YAML	prettier	81	2	0	3.43s
✅ YAML	v8r	23		0	58.0s
✅ YAML	yamllint	82		0	1.09s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by

[echoix]

So I added a skip comment for CKV_GHA_7, Check: CKV_GHA_7: "The build output cannot be affected by user parameters other than the build entry point and the top-level source location. GitHub Actions workflow_dispatch inputs MUST be empty. ".
https://docs.bridgecrew.io/docs/the-build-output-cannot-be-affected-by-user-parameters-other-than-the-build-entry-point-and-the-top-level-source-location-github-actions-workflow_dispatch-inputs-must-be-empty

Since it is not really a build, where we wouldn't want the output to change, here we want it to react to what the slash command dispatcher sends it, to know which comment to target, and to make sure the correct repo is called. Also, it is only users with write permissions that can call a workflow run.

[nvuillam]

I'm not sure I understand everything... plz could you add a section in contributing documentation, with examples ?

[echoix]

Ok, I'll write something soon. But it's for us, maintainers only, so that we could not require contributors to set up everything to run the ./build.sh. Only ask them to change the descriptors, and a quick comment to the PR with the following:

/build

will bring the PR up to date.

[echoix]

@nvuillam Is this enough?

And I added the following:

[echoix]

@nvuillam Is this enough to be merged? I'm quite eager to start using it soon :)

[nvuillam]

Just add a screenshot of a comment with the /build, and it's ok for me :)

[echoix]

For now there are 3 code blocks with the exact contents of what is placed.

@nvuillam Is this enough? And I added the following:

(Note that there is the "text" language in the top of the fence blocks added since these two screenshots)

To have a real use screenshot, it would have to either be already merged, or I re-setup a new repo with it merged into main. I tested in the beginning by committing to my branch and also main, and once sure that it works, I used the ref=<the-branch-name> to use the workflow of a branch. So I don't have a screenshot accessible now. You can see at the end of echoix#1 what it looks like.

[nvuillam]

Ok, you can add a screenshot in a next PR :)

[nvuillam]

Great , can't wait to see it in action :)

[echoix]

We'll be able to let users use the built-in web editor (pressing the period (.) key anywhere in github) to make simple edits, and fix the build without launching a real linux environment.

[echoix]

/help

Command	Description
/build	Updates the Dockerfile, documentation, and other files from the yml descriptors
/build [ref=...]	Same as /build, but executes workflow in any branch using the ref named argument. The reference can be a branch, tag, or a commit SHA. This can be useful to test workflows in PR branches before merging.
/help	Returns this help message

[nvuillam]

/help

Command	Description
/build	Updates the Dockerfile, documentation, and other files from the yml descriptors
/build [ref=...]	Same as /build, but executes workflow in any branch using the ref named argument. The reference can be a branch, tag, or a commit SHA. This can be useful to test workflows in PR branches before merging.
/help	Returns this help message

[echoix]

The 20 concurrent runners are already running, and I can't seem to find a way in github to specify priorities. It seems that they will be run only at the end of all the other builds

[nvuillam]

@echoix this is related to #2377 :/

[echoix]

@echoix this is related to #2377 :/

Yes and no, yes, since there are already running jobs so the 20 is supposed to be hit https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits
but no, since even with one Build & Deploy - DEV job on a branch (at one point, which doesn't have a matrix), other jobs where queued.

But its nothing like in the repos inside OSGeo, on good days (with a lot of contributions, across GDAL, PROJ (popular libraries in GIS), and grass), every 20 minute job has to wait 2h30 to start.

[echoix]

Here we go, the 2s+4s jobs ran :)