Updating 7.3.0 to 7.3.1

[YannickJadoul]

pypy -m ensurepip --default-pip is necessary since pip is already the version specified in requirements.txt and doesn't update, whereas before, the update would also install the pip script next to the pip3 and pip3.6 scripts.

See https://hub.docker.com/r/yannickjadoul/manylinux2010-pypy_x86_64 and pypa/cibuildwheel#304 for a temporary fork and cibuildwheel's test suite successfully using it.

[antocuni]

It looks to me that with this PR you are dropping support for PyPy 7.3.0, aren't you?

I think we never specified what is our policy towards micro versions, but e.g. this change would break pypy-wheels which hardcodes "7.3.0".

I am fine with saying that we only support the latest micro version for each major.minor, but we should at least document this explicitly I think

[mattip]

Why would this drop support for 7.3.0 ? They are meant to be ABI compatible

[YannickJadoul]

Argh, yes. I had indeed thought about ABI compatibility, but I hadn't thought of explicit references to the directory.

The problem with keeping both 7.3.0 and 7.3.1 is that the /opt/python/pp27-pypy_73 and /opt/python/pp36-pypy36_pp73 will be ambiguous.

[YannickJadoul]

Would renaming the 7.3.0/7.3.1 folders to 7.3 or 7.3.x be more robust for the future?

[antocuni]

Would renaming the 7.3.0/7.3.1 folders to 7.3 or 7.3.x be more robust for the future?

yes, I think this is the best thing to do. It will break pypy-wheels, but only once at least 😅

@mattip do we have any automated test which actually checks that wheels compiled for 7.3.0 works on 7.3.1?

[YannickJadoul]

yes, I think this is the best thing to do. It will break pypy-wheels, but only once at least

OK, let me see how easily I can do this :-)

[YannickJadoul]

Oh, that was a lot easier than I thought. No regex replacements or anything :-)
So, like this (7.3.x, indicating there is the patch version number), or do you prefer just 7.3?

Alternatively, I've just pushed another commit that keeps both 7.3.1 as well as a symlink 7.3.x to it. Or is that too many directories and links?

And while we're at it, /opt/python/pp27-pypy_41 is ambiguous as it both matches the ABI tag from 7.1.1 as 7.2.0, so the symlink points to /opt/pypy/pypy2.7-7.1.1 (as that's the first one to be installed) and there's no symlink in /opt/python pointing to /opt/pypy/pypy2.7-7.2.0. Or should some of the old versions be dropped by now?

[antocuni]

So, like this (7.3.x, indicating there is the patch version number), or do you prefer just 7.3?

I think I prefer 7.3.x, so it's more obvious that the actual version might change.

Alternatively, I've just pushed another commit that keeps both 7.3.1 as well as a symlink 7.3.x to it. Or is that too many directories and links?

Having 7.3.1 would be confusing, because that directory would vanish as soon as we build a newer manylinux image, wouldn't it?

And while we're at it, /opt/python/pp27-pypy_41 is ambiguous as it both matches the ABI tag from 7.1.1 as 7.2.0, so the symlink points to /opt/pypy/pypy2.7-7.1.1 (as that's the first one to be installed) and there's no symlink in /opt/python pointing to /opt/pypy/pypy2.7-7.2.0. Or should some of the old versions be dropped by now?

Good question. I think that so far the only user of pypy/manylinux is my pypywheels repo, which is using explicit version numbers, so in practice it doesn't matter :). I think it's fine to drop 7.1.1, though, we guarantee only the latest two releases anyway

[YannickJadoul]

Having 7.3.1 would be confusing, because that directory would vanish as soon as we build a newer manylinux image, wouldn't it?

Yes, but it would make clear which version is installed. And since #4 added unique IDs to tags, one could pin the version of the docker image, if depending on a certain version number.

But I don't mind either way. So I'll remove the symlink and the specific directory name?

Good question. I think that so far the only user of pypy/manylinux is my pypywheels repo, which is using explicit version numbers, so in practice it doesn't matter :). I think it's fine to drop 7.1.1, though, we guarantee only the latest two releases anyway

Well, and cibuildwheel, which is the immediate reason I'm making these PRs. But we only started supporting from 7.3.0 onwards, so that's fine for us as well!

It's also good to know there is a "two last (ABI) versions" policy on PyPy! That way, we can drop versions from cibuildwheel as well :-)

EDIT: 7.1.1 is still hosted from BitBucket (https://bitbucket.org/squeaky/portable-pypy/downloads), so it's good to drop the soon-to-be-dead link anyway.

[antocuni]

Yes, but it would make clear which version is installed. And since #4 added unique IDs to tags, one could pin the version of the docker image, if depending on a certain version number.
But I don't mind either way. So I'll remove the symlink and the specific directory name?

True enough. Actually, I don't have any specific opinion either. @mattip ?

It's also good to know there is a "two last (ABI) versions" policy on PyPy! That way, we can drop versions from cibuildwheel as well :-)

Ah no sorry, I was talking about my pypy-wheels. We don't have any official PyPy policy, so in general in the future we will probably want to keep old versions in the docker image for a while. What is the CPython policy for it?
But again, we are just starting to build/deploy wheels, so I think it's perfectly fine to drop old versions right now

[YannickJadoul]

What is the CPython policy for it?

I guess the difference with CPython is that CPython doesn't have distinction between Python version and implementation version (or at least patch releases don't change the ABI).

[antocuni]

I guess the difference with CPython is that CPython doesn't have distinction between Python version and implementation version (or at least patch releases don't change the ABI).

Ah, I see: for CPython, you have the precise version number in e.g. /opt/_internal/cpython-3.8.2 which is symlinked to /opt/cp38-cp38.

Ok, then I vote to do something similar for PyPy: we keep the explicit version number and call it pypy-7.3.1, and leave the ABI symlink for those who want a less precise version

[YannickJadoul]

Ok, then I vote to do something similar for PyPy: we keep the explicit version number and call it pypy-7.3.1, and leave the ABI symlink for those who want a less precise version

You mean to remove the 7.3.x folder again? Ok, I've removed those two commits.

[antocuni]

yes, sorry for the confusion and the change of mind, at first I didn't fully understand all the implications :(

Thank you for doing this, the PR LGTM. Feel free to merge!

[YannickJadoul]

yes, sorry for the confusion and the change of mind, at first I didn't fully understand all the implications :(

Thank you for doing this, the PR LGTM. Feel free to merge!

No problem at all! This was worth a discussion to make things future-proof indeed :-)

I don't have any rights to merge, though, but I'm happy with the changes. So do merge, if you want!

[antocuni]

Merged, and also added you to the group of people who can push :)

[YannickJadoul]

Hurray! (x2) Thanks :-) Let's get cibuildwheel up to speed, then :-)