Support running benchmarks on multiple long-standing branches of a repository

[punchagan]

Previously, we only ran the benchmarks on the default branch of a repository, apart from the pull requests. This PR adds support to run benchmarks on multiple branches of a repository, not just the default branch.

NOTE: This also includes the change in #428. That PR can either be merged before this one, or simply closed, in favor of this.

Closes #426

[ElectreAAS]

Overall the code seems good (bar my minor comments), but I am not able to properly test that it works.
I tried adding

{ "name": "ElectreAAS/merlin",
  "branches": [ "master", "mock-without-label" ]
}

to environment/development.conf, and adding commits to my mock repo, but the only thing that seems to run is the master branch.
What would be a better way to test it?

[punchagan]

Overall the code seems good (bar my minor comments), but I am not able to properly test that it works. I tried adding

{ "name": "ElectreAAS/merlin",
  "branches": [ "master", "mock-without-label" ]
}

to environment/development.conf, and adding commits to my mock repo, but the only thing that seems to run is the master branch. What would be a better way to test it?

This is how I tested it, too. Did you restart the containers after changing the development.conf file? Also, we ignore commits which are older than 2 weeks old, when running benchmarks (See Pipeline.filter_stale_repositories). I can't think of any other reason why you're unable to test it.

[punchagan]

Overall the code seems good (bar my minor comments), but I am not able to properly test that it works. I tried adding

{ "name": "ElectreAAS/merlin",
  "branches": [ "master", "mock-without-label" ]
}

to environment/development.conf, and adding commits to my mock repo, but the only thing that seems to run is the master branch. What would be a better way to test it?

This is how I tested it, too. Did you restart the containers after changing the development.conf file? Also, we ignore commits which are older than 2 weeks old, when running benchmarks (See Pipeline.filter_stale_repositories). I can't think of any other reason why you're unable to test it.

EDIT: I see that your commits on the branch are just about 2 weeks old, but also I see there's a "Pending" status from your local current-bench deployment on the latest commit on mock-without-label. So, not sure what exactly happened here, but might be worth looking at the logs again?

[ElectreAAS]

Thanks for the verification. With more recent commits everything runs fine indeed!
I'll consider this PR good to go when the other conversation is resolved :)

[ElectreAAS]

There was a CI failure that vanished when I rebuilt it, I have no clue what caused it.
I'll consider this good to go.

[punchagan]

Thanks for the careful review and merge, @ElectreAAS !