Does not recognize self-hosted-runner

[Borda]

Hello, not sure what or where is the problem correctly, but as we have defined some GH self-hosted runners, seems that they are not correctly recognized and the check fails with the following:

    $.jobs.help.runs-on: 'spot-self-hosted' is not valid under any of the given schemas
    $.jobs.help.runs-on: 'spot-self-hosted' is not one of ['macos-10.15', 'macos-11', 'macos-12', 'macos-latest', 'self-hosted', 'ubuntu-18.04', 'ubuntu-20.04', 'ubuntu-22.04', 'ubuntu-latest', 'windows-2019', 'windows-2022', 'windows-latest']
    $.jobs.help.runs-on: 'spot-self-hosted' is not of type 'array'
    $.jobs.help.runs-on: 'spot-self-hosted' does not match '^.*\\$\\{\\{(.|[\r\n])*\\}\\}.*$'

I was checking the schema and it seems to be supported but not sure... https://json.schemastore.org/github-workflow

[sirosen]

I'm looking at the diff between the latest schema version and the one released in v0.19.2 of check-jsonschema and I don't see anything which looks related. (You can see the compare view in github and look for the schema there.)

I am overdue to do a release -- I wanted to get a couple of minor features done and then ship one -- so we can try that and see if it helps, but I doubt it will.

To confirm: are you running the latest version? check-jsonschema --version or via pre-commit with rev: 0.19.2?
On the assumption that you are, I'll have to investigate how the schema from schemastore can be corrected. For self-hosted runners, runs-on can be any string? Or is it somehow constrained?

[Borda]

To confirm: are you running the latest version? check-jsonschema --version or via pre-commit with rev: 0.19.2?

yes, it uses:

Successfully installed check-jsonschema-0.19.2 click-8.1.3 importlib-resources-5.10.1 jsonschema-4.17.3 pkgutil-resolve-name-1.3.10 ruamel.yaml-0.17.21 ruamel.yaml.clib-0.2.7 zipp-3.11.0

For self-hosted runners, runs-on can be any string?

so far I know yes as you can define multiple different runners so the names shall be unique

[sirosen]

I just opened up the github-workflow schema from schemastore to see how this might be handled.

The issue, as I see it, is that runs-on: foo is valid, for almost any string foo. (Since any custom label could be used.)
So it's not possible to write a schema which forbids runs-on: foo, allows runs-on: windows-latest, and allows custom labels.
If we remove all validation of runs-on, then we won't be able to catch runs-on: macos-111 where macos-11 was intended.

However, the schema authors seem to have thought of this!

You can use a custom label as long as you put it in an array with self-hosted. According to the github docs, this applies multiple selectors; it won't make your job a matrix.
Therefore, the following workflow file should work okay:

name: self-hosted job
on:
  push:

jobs:
  myjob:
    runs-on: [self-hosted, spot-self-hosted]
    steps:
      - run: echo 'hi'

Are you open to this as a workaround?

I'm considering adding such a case to the check-jsonschema testsuite as a point of reference, and adding an FAQ page to the docs with this, but I'd like to confirm it really satisfies your case first.

[Borda]

You can use a custom label as long as you put it in an array with self-hosted

but what does this self-hosted do, won't it fail as this runner does not exist?

UPDATE: would it be possible to pass some extension for the validation schema, such as adding extra runner's names which would be validated alongside the built-ins?

[Borda]

@sirosen thank you, the runs-on: [self-hosted, spot-self-hosted] solves my problem so feel free to close it... 🦦

[sirosen]

Glad to hear it works!
I'm marking this as a docs issue. When I get to it, I'd like to put up an FAQ page and put this case in there. IMO, it's pretty non-obvious that you have to write a workflow this way to pass the schema.

[sirosen]

I've just put the example I gave above into the docs as part of an FAQ page, so it should be up in the stable docs as soon as I cut a release (in progress).