Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove arm64v8 #69

Merged
merged 1 commit into from
Mar 1, 2022
Merged

Remove arm64v8 #69

merged 1 commit into from
Mar 1, 2022

Conversation

tianon
Copy link
Member

@tianon tianon commented Mar 1, 2022

On many builds, we get:

pypy3: error while loading shared libraries: libffi.so.6: cannot open shared object file: No such file or directory

Which we can't resolve on Bullseye+ (same as s390x).

On others, we get:

pypy: error while loading shared libraries: libtinfow.so.6: cannot open shared object file: No such file or directory

Which appears to be a CentOS-specific library name and I couldn't convince pypy to be satisfied by any symlinks to either libtinfo or libtic (which were the suggestions I found on other projects for fixing similar linking issues).

See also #58 (for where we removed s390x bullseye builds).

Diff:
$ diff -u <(bashbrew cat pypy) <(bashbrew cat <(./generate-stackbrew-library.sh))
--- /dev/fd/63	2022-03-01 11:02:51.922171057 -0800
+++ /dev/fd/62	2022-03-01 11:02:51.922171057 -0800
@@ -3,22 +3,22 @@
 
 Tags: 3.9-7.3.8-bullseye, 3.9-7.3-bullseye, 3.9-7-bullseye, 3.9-bullseye
 SharedTags: 3.9-7.3.8, 3.9-7.3, 3.9-7, 3.9
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: fdb8ed8f0e8a7efbc8997841914264f126683f0e
 Directory: 3.9/bullseye
 
 Tags: 3.9-7.3.8-slim, 3.9-7.3-slim, 3.9-7-slim, 3.9-slim, 3.9-7.3.8-slim-bullseye, 3.9-7.3-slim-bullseye, 3.9-7-slim-bullseye, 3.9-slim-bullseye
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: fdb8ed8f0e8a7efbc8997841914264f126683f0e
 Directory: 3.9/slim-bullseye
 
 Tags: 3.9-7.3.8-buster, 3.9-7.3-buster, 3.9-7-buster, 3.9-buster
-Architectures: amd64, arm64v8, i386, s390x
+Architectures: amd64, i386, s390x
 GitCommit: fdb8ed8f0e8a7efbc8997841914264f126683f0e
 Directory: 3.9/buster
 
 Tags: 3.9-7.3.8-slim-buster, 3.9-7.3-slim-buster, 3.9-7-slim-buster, 3.9-slim-buster
-Architectures: amd64, arm64v8, i386, s390x
+Architectures: amd64, i386, s390x
 GitCommit: fdb8ed8f0e8a7efbc8997841914264f126683f0e
 Directory: 3.9/slim-buster
 
@@ -31,22 +31,22 @@
 
 Tags: 3.8-7.3.8-bullseye, 3.8-7.3-bullseye, 3.8-7-bullseye, 3.8-bullseye, 3-7.3.8-bullseye, 3-7.3-bullseye, 3-7-bullseye, 3-bullseye, bullseye
 SharedTags: 3.8-7.3.8, 3.8-7.3, 3.8-7, 3.8, 3-7.3.8, 3-7.3, 3-7, 3, latest
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: d26c7d6cf69b72d94ead67cae9fed40fd989129a
 Directory: 3.8/bullseye
 
 Tags: 3.8-7.3.8-slim, 3.8-7.3-slim, 3.8-7-slim, 3.8-slim, 3-7.3.8-slim, 3-7.3-slim, 3-7-slim, 3-slim, slim, 3.8-7.3.8-slim-bullseye, 3.8-7.3-slim-bullseye, 3.8-7-slim-bullseye, 3.8-slim-bullseye, 3-7.3.8-slim-bullseye, 3-7.3-slim-bullseye, 3-7-slim-bullseye, 3-slim-bullseye, slim-bullseye
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: d26c7d6cf69b72d94ead67cae9fed40fd989129a
 Directory: 3.8/slim-bullseye
 
 Tags: 3.8-7.3.8-buster, 3.8-7.3-buster, 3.8-7-buster, 3.8-buster, 3-7.3.8-buster, 3-7.3-buster, 3-7-buster, 3-buster, buster
-Architectures: amd64, arm64v8, i386, s390x
+Architectures: amd64, i386, s390x
 GitCommit: d26c7d6cf69b72d94ead67cae9fed40fd989129a
 Directory: 3.8/buster
 
 Tags: 3.8-7.3.8-slim-buster, 3.8-7.3-slim-buster, 3.8-7-slim-buster, 3.8-slim-buster, 3-7.3.8-slim-buster, 3-7.3-slim-buster, 3-7-slim-buster, 3-slim-buster, slim-buster
-Architectures: amd64, arm64v8, i386, s390x
+Architectures: amd64, i386, s390x
 GitCommit: d26c7d6cf69b72d94ead67cae9fed40fd989129a
 Directory: 3.8/slim-buster
 
@@ -59,22 +59,22 @@
 
 Tags: 3.7-7.3.8-bullseye, 3.7-7.3-bullseye, 3.7-7-bullseye, 3.7-bullseye
 SharedTags: 3.7-7.3.8, 3.7-7.3, 3.7-7, 3.7
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: 1520fc1d0d4d141c70504e6c6b8065b346aea1df
 Directory: 3.7/bullseye
 
 Tags: 3.7-7.3.8-slim, 3.7-7.3-slim, 3.7-7-slim, 3.7-slim, 3.7-7.3.8-slim-bullseye, 3.7-7.3-slim-bullseye, 3.7-7-slim-bullseye, 3.7-slim-bullseye
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: 1520fc1d0d4d141c70504e6c6b8065b346aea1df
 Directory: 3.7/slim-bullseye
 
 Tags: 3.7-7.3.8-buster, 3.7-7.3-buster, 3.7-7-buster, 3.7-buster
-Architectures: amd64, arm64v8, i386, s390x
+Architectures: amd64, i386, s390x
 GitCommit: 1520fc1d0d4d141c70504e6c6b8065b346aea1df
 Directory: 3.7/buster
 
 Tags: 3.7-7.3.8-slim-buster, 3.7-7.3-slim-buster, 3.7-7-slim-buster, 3.7-slim-buster
-Architectures: amd64, arm64v8, i386, s390x
+Architectures: amd64, i386, s390x
 GitCommit: 1520fc1d0d4d141c70504e6c6b8065b346aea1df
 Directory: 3.7/slim-buster
 
@@ -87,22 +87,22 @@
 
 Tags: 2.7-7.3.8-bullseye, 2.7-7.3-bullseye, 2.7-7-bullseye, 2.7-bullseye, 2-7.3.8-bullseye, 2-7.3-bullseye, 2-7-bullseye, 2-bullseye
 SharedTags: 2.7-7.3.8, 2.7-7.3, 2.7-7, 2.7, 2-7.3.8, 2-7.3, 2-7, 2
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: 4d46082ae0ff2771a80e4089ae9e58916134f2b1
 Directory: 2.7/bullseye
 
 Tags: 2.7-7.3.8-slim, 2.7-7.3-slim, 2.7-7-slim, 2.7-slim, 2-7.3.8-slim, 2-7.3-slim, 2-7-slim, 2-slim, 2.7-7.3.8-slim-bullseye, 2.7-7.3-slim-bullseye, 2.7-7-slim-bullseye, 2.7-slim-bullseye, 2-7.3.8-slim-bullseye, 2-7.3-slim-bullseye, 2-7-slim-bullseye, 2-slim-bullseye
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: 4d46082ae0ff2771a80e4089ae9e58916134f2b1
 Directory: 2.7/slim-bullseye
 
 Tags: 2.7-7.3.8-buster, 2.7-7.3-buster, 2.7-7-buster, 2.7-buster, 2-7.3.8-buster, 2-7.3-buster, 2-7-buster, 2-buster
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: 4d46082ae0ff2771a80e4089ae9e58916134f2b1
 Directory: 2.7/buster
 
 Tags: 2.7-7.3.8-slim-buster, 2.7-7.3-slim-buster, 2.7-7-slim-buster, 2.7-slim-buster, 2-7.3.8-slim-buster, 2-7.3-slim-buster, 2-7-slim-buster, 2-slim-buster
-Architectures: amd64, arm64v8, i386
+Architectures: amd64, i386
 GitCommit: 4d46082ae0ff2771a80e4089ae9e58916134f2b1
 Directory: 2.7/slim-buster

On many builds, we get:

> `pypy3: error while loading shared libraries: libffi.so.6: cannot open shared object file: No such file or directory`

Which we can't resolve on Bullseye+ (same as s390x).

On others, we get:

> `pypy: error while loading shared libraries: libtinfow.so.6: cannot open shared object file: No such file or directory`

Which appears to be a CentOS-specific library name and I couldn't convince `pypy` to be satisfied by any symlinks to either `libtinfo` or `libtic` (which were the suggestions I found on other projects for fixing similar linking issues).
@tianon tianon mentioned this pull request Mar 1, 2022
@yosifkit yosifkit merged commit 668b905 into docker-library:master Mar 1, 2022
@yosifkit yosifkit deleted the arm64v8 branch March 1, 2022 19:23
docker-library-bot added a commit to docker-library-bot/official-images that referenced this pull request Mar 1, 2022
Changes:

- docker-library/pypy@668b905: Merge pull request docker-library/pypy#69 from infosiftr/arm64v8
- docker-library/pypy@a4fbdd3: Remove arm64v8
docker-library-bot added a commit to docker-library-bot/official-images that referenced this pull request Mar 1, 2022
Changes:

- docker-library/pypy@c4bd7bb: Merge pull request docker-library/pypy#70 from infosiftr/sqlite3
- docker-library/pypy@fc08343: Refactor templating for better consistency
- docker-library/pypy@c3b9e88: Rebuild sqlite3 against the system-provided version
- docker-library/pypy@668b905: Merge pull request docker-library/pypy#69 from infosiftr/arm64v8
- docker-library/pypy@a4fbdd3: Remove arm64v8
@mattip
Copy link

mattip commented Mar 7, 2022

If you are using the tarballs from https://downloads.python.org/pypy/, there was a problem with the v7.3.8 release for aarch64. As you can see at that page, the correct download is https://downloads.python.org/pypy/pypy3.7-v7.3.8-aarch64-portable.tar.bz2, which properly packages the libraries needed. The tarball without portable was mis-built and does not have the needed libraries.

I would suggest you utilize the json file of downloads from https://downloads.python.org/pypy/versions.json to determine which files are the "latest" for each python version.

@tianon
Copy link
Member Author

tianon commented Mar 7, 2022

Oh nice, thanks @mattip!

It's probably a stretch, but any chance of getting the checksums in that JSON too? 😇

@tianon
Copy link
Member Author

tianon commented Mar 7, 2022

I'm also a bit confused about stable vs latest_pypy, but that's more of a curiosity -- it appears there are more with the set of both of those set to true than I would expect, so I think I'm misunderstanding what latest_pypy means in this context. 😅

$ wget -qO- 'https://downloads.python.org/pypy/versions.json' | jq 'map(select(.stable and .latest_pypy)) | map(del(.files))'
[
  {
    "pypy_version": "7.3.8",
    "python_version": "3.9.10",
    "stable": true,
    "latest_pypy": true,
    "date": "2022-02-19"
  },
  {
    "pypy_version": "7.3.8",
    "python_version": "3.8.12",
    "stable": true,
    "latest_pypy": true,
    "date": "2022-02-19"
  },
  {
    "pypy_version": "7.3.8",
    "python_version": "3.7.12",
    "stable": true,
    "latest_pypy": true,
    "date": "2022-02-19"
  },
  {
    "pypy_version": "7.3.8",
    "python_version": "2.7.18",
    "stable": true,
    "latest_pypy": true,
    "date": "2022-02-19"
  },
  {
    "pypy_version": "7.3.5",
    "python_version": "3.7.10",
    "stable": true,
    "latest_pypy": true,
    "date": "2021-05-23"
  },
  {
    "pypy_version": "7.3.3",
    "python_version": "3.6.12",
    "stable": true,
    "latest_pypy": true,
    "date": "2020-11-21"
  },
  {
    "pypy_version": "7.3.3",
    "python_version": "3.7.9",
    "stable": true,
    "latest_pypy": true,
    "date": "2020-11-21"
  },
  {
    "pypy_version": "7.3.2",
    "python_version": "3.6.9",
    "stable": true,
    "latest_pypy": true,
    "date": "2020-09-25"
  },
  {
    "pypy_version": "7.3.2",
    "python_version": "2.7.13",
    "stable": true,
    "latest_pypy": true,
    "date": "2020-09-25"
  }
]

(I would've expected a set of 4 here, not 9 😇 🙈)

@tianon
Copy link
Member Author

tianon commented Mar 7, 2022

Ah, found it in https://github.com/mozillazg/pypy/blob/daab65eb5779f127e616053b995223767ee0d603/pypy/doc/how-to-release.rst#other-steps:

Note the "latest_pypy" attribute: it is per-python-version. So if the new release overrides a current latest_pypy (both are 2.7.18, for instance), you must find the older version and set its "lastest_pypy" to "false" or check_versions.py (and the various tools) will fail.

@mattip
Copy link

mattip commented Mar 7, 2022

getting the checksums

I was taught not to put the checksums in the same place with the files, so they are at https://www.pypy.org/checksums.html.

@mattip
Copy link

mattip commented Mar 7, 2022

BTW, the versions.json was set up to be used by the setup-python github action so there may be some documentation and code you can use there (by searching for pypy or versions.json). I can add more fields if you can define what would make your life easier.

@tianon
Copy link
Member Author

tianon commented Mar 7, 2022

Thanks! I'll work on a fix to consume that file appropriately. I appreciate the pointer and the help. 😁

The only other thing I can think of is "beta" vs "non-beta" of a particular Python version within a given release -- right now I'm relying on the blog posts, where the latest clearly states that 3.9 is still considered "beta" but 3.8 is now stable. 😄

@mattip
Copy link

mattip commented Mar 7, 2022

I hope using the file makes your life easier, and will help automate updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants