Handling different OSes

[ForeverWintr]

It seems like github does not provide all python versions on all OSes. For example, currently the version-manifest only has 3.9.15 for linux.

As a result, I'm having trouble mixing latest-python-versions with an OS matrix. Is there a known solution to this? Here is the relevant section of my workflow:

  get-python-versions:
    runs-on: ubuntu-latest
    outputs:
      python-matrix: ${{ steps.get-python-versions-action.outputs.latest-python-versions }}
    steps:
    - uses: snok/latest-python-versions@v1.1.2
      id: get-python-versions-action
      with:
        min-version: 3.9
        include-prereleases: true

  test:
    needs: [lint, get-python-versions]
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        python-version: ${{ fromJson(needs.get-python-versions.outputs.python-matrix) }}
        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
    steps:
    - uses: actions/checkout@v3

    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4.2.0
      with:
        python-version: ${{ matrix.python-version }}

[sondrelg]

Hmm I see 👍 I guess we could rewrite the action to produce a latest version per OS. Would you be interested in opening a PR for that?

[ForeverWintr]

I'd be happy to give it a try, although I'm not very familiar with github actions yet. Can you suggest what format would be best, and how it could be accessed in the actions yaml?

Another option I thought of is to simply drop the patch version (returning 3.9 or 3.9.* instead of 3.9.15). Do you think that's worth exploring?

[sondrelg]

There's a few reasons why including the patch version is a good idea, in general. When you build a cache key using the minor version you might build, e.g., a venv on one patch version of Python then see problems when the github runner upgrades to another (and the cache isn't invalidated). I'm sure there are other examples.

This page describes job outputs pretty well: https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs. Right now we output a latest-python-versions, but I guess we could also add another output called latest-python-versions-per-os that's structured {"windows": "3.11.0", "linux": "3.11.0", "macos": "3.11.0"} (or whatever makes sense based on the source data).

Does that make sense, or did you have something else in mind? We could also have changed the existing output, but I'm not sure there's a good reason to break backwards compatibility here 🙂

[ForeverWintr]

Thanks for the explanation, it makes sense.

It looks like this is more complicated than just using "windows" or "linux" as keys however, as even within a given platform the versions available are not always consistent. For example, 3.6.15 is available on ubuntu-20 but not ubuntu-22.

I think the approach that makes most sense is to return something like this:

{'ubuntu-22.04': ['3.11.0', '3.10.8', '3.9.15'],
 'ubuntu-20.04': ['3.11.0', '3.10.8', '3.9.15'],
 'macos-12': ['3.11.0', '3.10.8', '3.9.14']}

But I haven't yet found a way to map from the platform/version/arch labels in the version manifest and the runner image labels. Are you aware of one?

For posterity, here are the current latest non EOL versions for each OS:

{'darwin-x64': ['3.12.0-alpha.2',
                '3.11.0',
                '3.10.8',
                '3.9.14',
                '3.8.14',
                '3.7.15'],
 'linux-18.04-x64': ['3.12.0-alpha.2',
                     '3.11.0',
                     '3.10.8',
                     '3.9.15',
                     '3.8.14',
                     '3.7.15'],
 'linux-20.04-x64': ['3.12.0-alpha.2',
                     '3.11.0',
                     '3.10.8',
                     '3.9.15',
                     '3.8.14',
                     '3.7.15'],
 'linux-22.04-x64': ['3.12.0-alpha.2',
                     '3.11.0',
                     '3.10.8',
                     '3.9.15',
                     '3.8.14',
                     '3.7.15'],
 'win32-x64': ['3.12.0-alpha.2',
               '3.11.0',
               '3.10.8',
               '3.9.13',
               '3.8.10',
               '3.7.9'],
 'win32-x86': ['3.12.0-alpha.2',
               '3.11.0',
               '3.10.8',
               '3.9.13',
               '3.8.10',
               '3.7.9'],
 'linux-16.04-x64': ['3.10.0-beta.3', '3.9.5', '3.8.9', '3.7.10']}

[sondrelg]

Right, arch also matters 🤕 I guess that will make it hard to generalize on runner OS and version actually.

There's no arm yet, but I think it's on Github's roadmap now that Azure supports it