CI: improve coverage of self-hosted runners

[delan]

our self-hosted runners are fast, but there are a lot of cases where they aren’t used yet. here’s an overview:

• events that trigger workflows
  • new commits on main (push)
  • merge queue (merge_group)
  • try labels (workflow_dispatch)
  • mach try, except when targeting a fork (push)
  • pull request checks (pull_request)
    • ci: Tokenless self-hosted runner select #39900
• kinds of builds
  • production jobs
    • bake a prebuilt servo into our images
  • android build jobs
  • ci: Run bencher jobs on self-hosted runners #39269
  • Add unittest code coverage to CI  #39789
  • ci: Run lint jobs on self-hosted runners #39915
  • 10-minute wpt jobs Add support for running WPT suite ci-runners#21
  • libservo builds cargo build -p libservo is untested in CI #38015
  • macOS arm64 builds
• builds with less impact on developer experience
  • nightly release builds CI: disable self-hosted runners for nightly.yml #34982

[sagudev]

pull request checks (pull_request)
blocked on eliminating the use of secrets (MONITOR_API_TOKEN)

IIRC, this is also security feature as for second contribution workflow runs are run automatically which can be security issue (given persistent state between runs).

[jschwe]

Do we have any statistics about self hosted runner utilisation (how much time they are idle vs running jobs)?
Edit: I'd also be interested in how many reservation requests are made vs. how many can be fulfilled.

[delan]

yes! see ci-runners/analysis/usage. i’m keen to see if this utilisation improves once pull request jobs land in #39900.