[scheduler] Add backfilling #1776
Conversation
app_dart/lib/src/service/config.dart
Outdated
| @@ -46,7 +46,7 @@ class Config { | |||
| /// This adds support for the `waiting for tree to go green label` to the repo. | |||
| /// | |||
| /// Relies on the GitHub Checks API being enabled for this repo. | |||
| static Set<RepositorySlug> supportedRepos = <RepositorySlug>{ | |||
| Set<RepositorySlug> supportedRepos = <RepositorySlug>{ | |||
There was a problem hiding this comment.
This needed to be changed so tests could override it. Otherwise, the data would look weird where there would be 6 backfills instead of 1 (1 for each supported repo)
| return null; | ||
| } | ||
|
|
||
| final List<FullTask> backfillTask = |
There was a problem hiding this comment.
It would be interesting to use this logic to rerun failed tests to surface flakes. Not needed in this PR though.
| late FakePubSub pubsub; | ||
| late FakeScheduler scheduler; | ||
|
|
||
| group('BatchBackfiller', () { |
There was a problem hiding this comment.
could you add tests for batch scheduling cases: more than 1 target are scheduled when they all need to schedule; only one target is scheduled when itself needs to schedule, etc?
There was a problem hiding this comment.
Thanks, good point on having multiple columns being tested at a time.
To clarify, I added tests for when:
- Only one column needs to be backfilled
- 2 columns need to be backfilled
| @@ -27,6 +27,10 @@ cron: | |||
| url: /api/vacuum-github-commits | |||
| schedule: every 1 hours | |||
|
|
|||
| - description: backfills builds | |||
| url: /api/scheduler/batch-backfiller | |||
| schedule: every 5 minutes | |||
There was a problem hiding this comment.
I believe this is a no-op for LUCI triggered builders?
There was a problem hiding this comment.
Correct. Those targets have a OmitPolicy which means Cocoon doesn't do anything.
CaseyHillers
added
the
autosubmit
flutter/flutter#100793
This implements a cron job that looks at task columns, and backfills them. It will only run a max of one task per column at a given time.
This is needed to migrate the devicelab tests over to the cocoon scheduler.