PR Bot

Experimental GitHub Bot to manage SharingCloud development workflows.
Uses PostgreSQL and Redis.

• CHANGELOG
• Project structure

Why ?

GitHub is great and contains many useful features, but sometimes you need more. Today, you can use labels to have a quick view on the steps of your workflow, but you have to put them by hand, or write scripts to manipulate them.

One GitHub-hosted possibility is to use GitHub Actions as a workflow orchestrator (listening on pull request updates, added comments, or even check runs), but GitHub Actions are checks, so they need to boot at each GitHub event you want to support. They will also show on your Actions list, so your actual actions (as tests or builds) are not easily visible anymore.

Another problem with GA is that you need to duplicate your workflows for each of your repositories, even if you can use packages (as on GitHub Packages).

So here is another idea, why not use a centralized bot, which can react to webhook events to manage your workflows, can handle multiple repositories at once, maintaining state in a database and using per-repository configuration files ? Using the workflow events and GitHub API you can have almost real-time interactions.

Some bots like this already exists, like the Bors bot for the Rust programming language, but this bot will be more specifically designed for our own use-cases. Il will add some features:

• Use a command system through issue comments to drive the bot
• Specify a regular expression to be matched for pull request titles
• Specify manual QA status for a pull request (skipped, fail, pass, waiting)
• Specify required reviewers for a particular pull request, and lock the merge if these reviews do not pass
• Apply specific labels on pull requests depending on their current state
• Maintain an automatically updated status comment with relevant details
• And maintain a commit status to prevent or allow merge

It will also be drivable via a command-line interface (for one-time actions as synchronizing state, or import/export), and if possible a terminal-like user interface (TUI, à la htop).

Features

• Manage PR lifecycle with local data stored in a PostgreSQL database
• Reacts to GitHub Webhooks to update review status
• Generate a summary comment (once per PR)
• Validate PR titles depending on per-repository regexes
• Reacts to comments: Set QA status (or skip), ping, lock/unlock, merge, etc.
• Require mandatory reviewers
• Merge support with merge rules depending on head and base branches (specific merge strategies)
• Enable auto-merge
• Actions that can be triggered from external sources, with simple token-based authentication
• Give rights to external sources on specific repositories
• Send GIFs!
• Terminal UI interface to manage pull request status

Infrastructure schema

Step labels

Process can be followed with labels, which are auto applied depending on the current pull request state, in this order:

• PR is WIP? step/wip
• PR title is not valid? step/awaiting-changes
• Waiting for checks? step/awaiting-checks
• Checks failed? step/awaiting-changes
• Waiting for required reviews? step/awaiting-required-review
• Waiting for reviews? step/awaiting-review
• Waiting for QA? step/awaiting-qa
• QA failed? step/awaiting-changes
• PR is locked? step/locked
• All good? step/awaiting-merge

Available message commands

This README supposes the default bot handle: 'bot'.

• bot noqa+: Skip QA validation
• bot noqa-: Enable QA validation
• bot qa+: Mark QA as passed
• bot qa-: Mark QA as failed
• bot qa?: Mark QA as waiting
• bot nochecks+: Skip checks validation
• bot nochecks-: Enable checks validation
• bot automerge+: Enable auto-merge for this PR (once all checks pass)
• bot automerge-: Disable auto-merge for this PR
• bot lock+ <reason?>: Lock a pull-request (block merge)
• bot lock- <reason?>: Unlock a pull-request (unblock merge)
• bot r+ <reviewers>: Assign reviewers (you can assign multiple reviewers)
• bot req+ <reviewers>: Assign required reviewers (you can assign multiple reviewers)
• bot r- <reviewers>: Unassign reviewers (you can unassign multiple reviewers)
• bot strategy+ <strategy>: Override merge strategy for this pull request
• bot strategy-: Remove the overriden merge strategy for this pull request
• bot merge <merge|squash|rebase?>: Try merging the pull request with optional strategy
• bot labels+ <label>: Set specific labels
• bot labels- <label>: Unset specific labels
• bot ping: Ping me
• bot gif <search>: Post a random GIF with a tag
• bot is-admin: Check if you are admin
• bot help: Show this comment

Available admin message commands

If you have admin rights (you can set with auth add-admin-rights <username>), you have access to the following commands:

• bot admin-help: Show this comment
• bot admin-enable: Enable me on a pull request with manual interaction
• bot admin-disable: Disable me on a pull request with manual interaction
• bot admin-add-merge-rule <base> <head> <strategy>: Add/Update a merge rule for this repository
• bot admin-set-default-needed-reviewers <count>: Set default needed reviewers count for this repository
• bot admin-set-default-merge-strategy <merge|squash|rebase>: Set default merge strategy for this repository
• bot admin-set-default-pr-title-regex <regex?>: Set default PR title validation regex for this repository
• bot admin-set-default-automerge+: Set automerge enabled for this repository
• bot admin-set-default-automerge-: Set automerge disabled for this repository
• bot admin-set-default-qa-status+: Enable QA validation by default for this repository
• bot admin-set-default-qa-status-: Disable QA validation by default for this repository
• bot admin-set-default-checks-status+: Enable checks validation by default for this repository
• bot admin-set-default-checks-status-: Disable checks validation by default for this repository
• bot admin-set-needed-reviewers <count>: Set needed reviewers count for this PR
• bot admin-reset-reviews: Reset and update reviews on pull request (maintenance-type command)
• bot admin-reset-summary: Create a new summary message (maintenance-type command)
• bot admin-sync: Update status comment if needed (maintenance-type command)

Exposed URLs

• GET /: Index route, display a welcome message
• POST /webhook: Entrypoint for GitHub webhooks
• POST /external/set-qa-status: Update QA status from external source (e.g. JIRA), needs a JWT
• GET /health: Health check, with a field for PostgreSQL and another for Redis
• GET /metrics: A set of Prometheus metrics

Building and developing

This project is written in the Rust programming language, so to build you have to install the Rust tools. To use the development server with "watch mode", you will also need the cargo-watch tool (installable using cargo install cargo-watch).
You will also need the just command runner.

On Ubuntu, to build locally, you need the following packages:

• pkg-config
• libssl-dev
• libpq-dev

Running server locally

Type just runserver.

Docker building

You can type just docker-build to automatically generate a Docker image with the current bot version.

Once your image is ready, you can use the docker/docker-compose.yml file to easily mount a Docker Compose stack.

GitHub installation

Webhook only (using a standard account)

• Add a Webhook on GitHub,
• Use http(s)://[your-domain]/webhook as Payload URL,
• Set the Content Type as application/json,
• Use a secret if needed (configure your bot with the BOT_GITHUB_WEBHOOK_SECRET env. var.),
• Then, enable the following events:
  • Check suite,
  • Issue comment,
  • Pull request,
  • Pull request review,
  • Pull request review comment

Once configured, you should receive a ping event.

GitHub application (using a bot account)

• Go to your Account, in "Settings" > "Developer settings" > "GitHub Apps" > "New GitHub App",
• Set the app name you want, the homepage URL you want,
• Enable "Webhooks", and set the "Webhook URL" http(s)://[your-domain]/webhook,
• Use a secret if needed (configure your bot with the BOT_GITHUB_WEBHOOK_SECRET env. var.),
• Setup the required "Repository permissions":
  • Checks: Read/Write (to read/set checks),
  • Commit statuses: Read/Write (to read/set statuses),
  • Contents: Read/Write (to merge the pull request in the end),
  • Issues: Read/Write (to read/set labels),
  • Metadata: Read-only (well it's mandatory and enabled by default),
  • Pull requests: Read/Write (to read/post/edit comments, get PR info).
• Setup "Subscribe to events":
  • Check suite,
  • Issue comment,
  • Pull request,
  • Pull request review,
  • Pull request review comment.
• Create the GitHub App, and keep the "App ID" (shown at the top of your app page, set it as the BOT_GITHUB_APP_ID env. var.),
• Generate a "private key" for your app, using the button available on your app page, GitHub will make you download the key on your computer,
  • Then copy its content in your BOT_GITHUB_APP_PRIVATE_KEY env. var. (if you are using an environment file, like a .env file, you have to put the key between double quotes (") and replace newlines by the "\n" character)
• Now, install the GitHub App on your repositories (or specific repositories), and when you will be on the "installation page" look at the url, which should be like https://github.com/[your account]/settings/installations/[installation_id], and copy that "installation ID" from the URL to the BOT_GITHUB_APP_INSTALLATION_ID env. var.
• And that's it, your bot should be working !