[New builder] Introduce Cancelot

[pavlospt]

Background

After trying out CloudBuild we needed a way to cancel any running builds for the same branch when we were pushing a new commit, in order to avoid billing extra not needed minutes. Since CloudBuild does not offer an out of the box solution we built Cancelot.

How it works

Cancelot will be invoked when the build starts and will try to find any running jobs that match the following filter:

build_id != "[CURRENT_BUILD_ID]" AND 
source.repo_source.branch_name = "[BRANCH_NAME]" AND 
status = "WORKING" AND 
start_time<"[CURRENT_BUILD_START_TIME]"

Usage

Add the builder as the first step in your project's cloudbuild.yaml:

steps:
- name: 'gcr.io/$PROJECT_ID/cancelot'
  args: [ 
    '--current_build_id', '$BUILD_ID',
    '--branch_name', '$BRANCH_NAME'
  ]

[rharter]

This looks awesome, I'm installing from your fork today!

[rharter]

Should this be in here?

[pavlospt]

Hmm not 100% sure, just added it cause I was using Goland for it!

[sanastos]

Although I can't imagine we would want to check in intellij config directories, there's no extant gitignore so I'd leave it out of this commit.

[pavlospt]

@rharter Hey, sry for pinging you, but do you maybe know if there is anyone from Google checking this repo? 😞 It seems like there has been some inactivity.

[rharter]

Hey @nof20, I've tested this out and it works great (and fixes a pain point I've had for some time, causing me to manually cancel builds when I discover 🤦‍♂️ errors when reviewing the diff while posting a PR). Any chance we can push it along? I'm happy to help how I can.

[viktorvoltaire]

A feature request for cancelot, could you add the option to look for a specific tag instead of branch?

It seems like source.repo_source.branch_name is not getting populated when using the GCB github app to trigger builds. My workaround for this would be to specify the branch name as a tag

[pavlospt]

@viktorvoltaire sure, this is something that could be done! How would you expect it to work through? Have either Branch name or tag expected and terminate in case none is declared or both are declared?

[viktorvoltaire]

Well as long as branch_name is not mandatory, since it can't be filtered on when using the github app, im happy :)
These are the flags i would run it with --current_build_id $BUILD_ID --tag $BRANCH_NAME

[pavlospt]

Will see what I can have ready in the end of the week !

[viktorvoltaire]

@pavlospt unfortunately my feature request might not be valid due to limitations in cloudbuild. One cant have special characters like / in tags so if you have a branch like feature/add_character cloudbuild will fail with invalid build tag "feature/add_character": must match format "^([\\w][\\w.-]{0,127})$"

[pavlospt]

@viktorvoltaire yes, this is indeed a limitation which caused some trouble for us as well.

[jthegedus]

Is it an issue to run a regex replace over the BRANCH_NAMEs before using them?

[viktorvoltaire]

I haven't been able to solve tags are specified and evaluated before running build steps and seems to be static. @jthegedus do you have a solution that works?

[pavlospt]

@viktorvoltaire there is source.repo_source.tag_name as well. Isn't that populated either?

[viktorvoltaire]

No that is empty.

[pavlospt]

@viktorvoltaire just to be sure I am up-to-date which Github app are you referring to?

[viktorvoltaire]

https://github.com/marketplace/google-cloud-build

[pavlospt]

@sanastos would it be possible to take a look at this one as well?

[sanastos]

This looks good to me, although I recommend adding an example that exercises this behavior (I guess two builds, one that hangs, one that cancels). This will reduce the likelihood that a future maintainer will accidentally break this.

[sanastos]

Although I can't imagine we would want to check in intellij config directories, there's no extant gitignore so I'd leave it out of this commit.

[pavlospt]

@sanastos I do not think that this is something doable, because the work done by Cancelot is based on different builds triggered with different UUIDs, which means there will be no way to check if things will break unfortunately :/

[pavlospt]

@sanastos I removed the .gitignore

[sanastos]

Ah, I see what you're saying. This only cancels triggered builds, and there's no easy way to fake that.

Well, as a bare minimum, how about a test like:

steps:
- name: 'gcr.io/$PROJECT_ID/cancelot'
  args: [
    '--current_build_id', '$BUILD_ID',
    '--branch_name', "$BRANCH_NAME"
  ]
- name: 'ubuntu'
  args: ['echo', 'done']

And then a README.md that instructs you to launch as:
gcloud builds submit . --config=cloudbuild.yaml --substitutions=BRANCH_NAME="test"

Alternatively just hard code a fake branch name in the cloudbuild.yaml.

This at least exercises the code a bit, better than nothing.

[pavlospt]

@sanastos sure will add your suggestions tomorrow then!

[sanastos]

Looks good, thanks for your contribution. And kudos on the cool name 💯

[Kampe]

Love the concept, but is there any way to specify which repository this should actually apply to? Right now it just cancels all running builds if a new one comes in no matter the source.

[pavlospt]

@Kampe this takes into account branch name. Do you happen to have duplicate branch names between different repos? If so please file it as an issue and i will prepare a PR to handle this case as well!