Ability to restrict runs-on #66

Want to help get this issue implemented? Donate to the action-validator code fund.

Intro

Hey, following recent changes, essentially any value of runs-on: is accepted. Since one of the outcomes of this action is to reduce the possibility of broken workflows, I think it would be very beneficial to validate runs-on somehow.

Proposal

• By default validate against freely available GitHub Hosted runners (no XL, no self-hosted)
  This way the validator brings immediate value to hopefully most users,
• With a CLI parameter (--allowed-runners) allow setting valid (incl. custom) values.
  but also gives the ability to override if needed.

Syntax

• Comma separated list
• Prefix @ for built-in (in the validator) groups
• Prefix - for exclusion
• No - prefix = positive listing

Built-in groups

(might be more):

• github-free: ubuntu-latest, ubuntu-22.04, etc.
• github-latest: ubuntu-latest, windows-latest, etc.
• github-xl: -xl and -cores suffixed ones
• self-hosted: special value for current 0.5.4 behaviour

Custom groups

If there's a known list of self hosted runners, it's possible to just list them all and use no groups. That is, to define a custom group, users need to list all runners explicitly.

For reusability the custom groups could be extracted into environment variables / organization level variables, like ACTION_VALIDATOR_ALLOWED_RUNNERS, and then that used in CLI with shell expansion: --allowed-runners=${ACTION_VALIDATOR_ALLOWED_RUNNERS},-ubuntu-latest.

Calculation

• Start with empty set
• Add all positive groups, and positive runners
• Remove all negative groups and runners

i.e. order of flags doesn't matter.

Examples

0. the default when no argument provided

--allowed-runners=!github-free

1. a GitHub Enterprise user might validate like this, if they allow 2 self-hosted runners, and want to be explicit about versions:

--allowed-runners=!github-free,@github-large,-@github-latest,my-special-runner1,my-special-runner2

2. an open-source community member GitHub user, who wants to use latest without automagic updates:

--allowed-runners=ubuntu-22.04,windows-2022,macos-12