Disable cache: 'pip' for unreleased Python versions

[pquentin]

Description:

Disable cache: 'pip' for unreleased Python versions (right now, that means only Python 3.11)

Justification:

The ABI between Python 3.11 versions is not stable, and in fact recently changed between Python 3.11 alpha 2 and alpha 3. This is why projects don't provide binary wheels for Python 3.11 until the ABI is stabilized. Unfortunately, when enabling pip's cache, this means that wheels cached for Python 3.11 alpha 2 will get used for Python 3.11 alpha 3 and are then likely to break or segfault. This is not a theoretical concern, this broke urllib3's test suite, and likely cryptography's test suite as well. See https://bugs.python.org/issue46320 for more details.

Without this change, our options are disabling pip's cache altogether or stop testing Python 3.11.

Are you willing to submit a PR?

No, sorry.

[dmitry-shibanov]

Hello @pquentin. Thank you for your report. We'll investigate it deeper.

[hugovk]

The pip cache can be especially useful for testing dev versions, because there are few wheels available and it can be very slow to build stuff from source. Far better to cache after the first time.

The ABI can change during the CPython alpha and beta phase. It's also possible during RC, but they aim not to.

So we need to make sure the cache isn't reused between say 3.11.0a2 and 3.11.0a3.

Onw way would be to include this fully qualified version number in the cache key.

Similar to #303 which includes x.y but use the full 3.11.0a3 form.

Thoughts?

[sethmlarson]

@hugovk This would solve our issue and the issue you linked to, sounds good to me! 🎉

[brettcannon]

Another option is to use sys.version as that can potentially cover PyPy nightly as well.

[dmitry-shibanov]

Hello everyone. We merged #303 and released new version of the action with these changes. Now cache will be restored for the particular python versions.
For now I'm going to close the issue. If you have any concerns feel free to ping us.

[hugovk]

Thank you, looks good! I confirm the cache key includes the Python x.y.z version and also the alpha part:

Cache saved with the key: setup-python-Linux-python-3.11.0-alpha.5-pip-9e9ed1f18f826732cf60110ffde7c496bfa3061029bfd944cd9739b9c2ef7185

https://github.com/hugovk/test/runs/5084690004?check_suite_focus=true#step:15:5

3.7 is x.y.z:

Unable to reserve cache with key setup-python-Linux-python-3.7.12-pip-9e9ed1f18f826732cf60110ffde7c496bfa3061029bfd944cd9739b9c2ef7185, another job may be creating this cache.

https://github.com/hugovk/test/runs/5084690124?check_suite_focus=true#step:15:2

(Aside: I see that latter didn't create its own cache because there's both ubuntu-20.04 and ubuntu-18.04 jobs resolving to Linux, but that's unrelated to this particular issue and may or may not actually be a problem :)