`_struct.cpython-310-darwin.so` is not a fat binary #439

charles-cooper · 2022-06-18T04:15:19Z

Description:
since actions/python-versions#114, the python binary is universal2. but apparently, some of the .so files are not.
https://github.com/vyperlang/vyper/runs/6945662103?check_suite_focus=true raises the following error

  File "/Users/runner/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/PyInstaller/utils/osx.py", line 312, in binary_to_target_arch
    raise IncompatibleBinaryArchError(f"{display_name} is not a fat binary!")
PyInstaller.utils.osx.IncompatibleBinaryArchError: /Users/runner/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so is not a fat binary!

i downloaded the build that was used in the action from https://github.com/actions/python-versions/releases/download/3.10.5-2452607758/python-3.10.5-darwin-x64.tar.gz, and inspected the files.
the main binary looks good!

$ file bin/python3.10
bin/python3.10: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>] [arm64:Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>]

however, the library in question does not:

$ file lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so 
lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so: Mach-O 64-bit x86_64 bundle, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL>

Action version:
v4

Platform:

Ubuntu
macOS
Windows

Runner type:

Hosted
Self-hosted

Tools version:
3.10.5

The text was updated successfully, but these errors were encountered:

bwoodsend · 2022-06-18T22:43:41Z

Looks like the entirety of lib/python3.10/ḷib-dynload is single arch.

~/D/python-3.10.5-darwin-x64> for file in bin/* **.so **.dylib
                                  echo $file (llvm-lipo -archs $file 2>/dev/null)
                              end
bin/2to3
bin/2to3-3.10
bin/idle3
bin/idle3.10
bin/pip3
bin/pip3.10
bin/pydoc3
bin/pydoc3.10
bin/python3 x86_64 arm64 
bin/python3-config
bin/python3-intel64 x86_64 
bin/python3.10 x86_64 arm64 
bin/python3.10-config
bin/python3.10-intel64 x86_64 
lib/python3.10/lib-dynload/_asyncio.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_bisect.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_blake2.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_bz2.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_cn.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_hk.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_iso2022.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_jp.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_kr.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_codecs_tw.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_contextvars.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_crypt.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_csv.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_ctypes.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_ctypes_test.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_curses.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_curses_panel.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_datetime.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_dbm.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_decimal.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_elementtree.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_gdbm.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_hashlib.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_heapq.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_json.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_lsprof.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_lzma.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_md5.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_multibytecodec.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_multiprocessing.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_opcode.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_pickle.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_posixshmem.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_posixsubprocess.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_queue.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_random.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_scproxy.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha1.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha3.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha256.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sha512.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_socket.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_sqlite3.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_ssl.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_statistics.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_struct.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testbuffer.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testcapi.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testimportmultiple.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testinternalcapi.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_testmultiphase.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_tkinter.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_uuid.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_xxsubinterpreters.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_xxtestfuzz.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/_zoneinfo.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/array.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/audioop.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/binascii.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/cmath.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/fcntl.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/grp.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/math.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/mmap.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/nis.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/pyexpat.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/readline.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/resource.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/select.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/syslog.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/termios.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/unicodedata.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/xxlimited.cpython-310-darwin.so x86_64 
lib/python3.10/lib-dynload/xxlimited_35.cpython-310-darwin.so x86_64 
lib/python3.10/ḷib-dynload/zlib.cpython-310-darwin.so x86_64 
lib/libpython3.10.dylib x86_64 arm64

panticmilos · 2022-06-19T13:59:28Z

Hi @charles-cooper,

Thank you for submitting the issue, we will take look into it.

charles-cooper · 2022-08-23T20:47:30Z

@panticmilos @bwoodsend bump -- any news on this?

panticmilos · 2022-09-14T14:03:36Z

Hi @charles-cooper, we have reverted logic here actions/python-versions#175, so I am going to close the issue.

And regarding this 3.10.5 version, it's not going to be using universal2 builds anyway since this PR actions/python-versions#184 will affect versions from 3.11.0-beta.1

Feel free to continue the conversation, reopen the issue, or open a new issue.

charles-cooper · 2022-09-14T18:15:25Z

@panticmilos i'm not sure i am following what the resolution is here. so universal2 builds of python3 are no longer allowed?

IvanZosimov · 2022-09-15T06:37:59Z

Hi, @charles-cooper 👋 Let me clarify the situation here - now python builds in universal2 are unavailable, because making them from source code leads to many unexpected problems as you, for example, mentioned in this issue. So, it was decided not to build python in universal2 and this PR revert the logic of building python. Now, we are going not to build python for macOS in universal2 on our own, but use prebuilt packages right from the python dist. This PR should add this functionality, and starting from the 3.11.0-beta.1 you will be able to use unversal2 python .

chore: update published build to python 3.11 and also update the rest of the CI to use 3.11. two reasons for doing this, even though 3.11 is a relatively new release: - python 3.11 is about 10% faster based on local benchmarks - universal2 images for darwin (on github actions) are only available for python builds starting with 3.11. cf. actions/setup-python#439 (comment)

and also update the rest of the CI to use 3.11. two reasons for doing this, even though 3.11 is a relatively new release: - python 3.11 is about 10% faster based on local benchmarks - universal2 images for darwin (on github actions) are only available for python builds starting with 3.11. cf. actions/setup-python#439 (comment)

charles-cooper added bug Something isn't working needs triage labels Jun 18, 2022

panticmilos removed the needs triage label Jun 19, 2022

charles-cooper mentioned this issue Jun 19, 2022

feat: build for aarch64 vyperlang/vyper#2687

Merged

LadyK-21 mentioned this issue Jun 20, 2022

[Snyk] Upgrade semver from 7.1.3 to 7.3.7 LadyK-21/setup-python#6

Open

panticmilos self-assigned this Jun 21, 2022

panticmilos assigned IvanZosimov Jun 29, 2022

This was referenced Aug 19, 2022

[Snyk] Upgrade semver from 7.1.3 to 7.3.7 47-studio-org/setup-python#8

Open

[Snyk] Upgrade semver from 7.1.3 to 7.3.7 devcode1981/setup-python#8

Open

[Snyk] Upgrade semver from 7.1.3 to 7.3.7 MarcelRaschke/setup-python#10

Open

panticmilos closed this as completed Sep 14, 2022

chidiwilliams mentioned this issue Oct 3, 2022

Won't run on mac M1 chidiwilliams/buzz#6

Closed

Neizvestnyj mentioned this issue Dec 25, 2022

Is not a fat binary! with target_arch="universal2" pyinstaller/pyinstaller#7354

Closed

charles-cooper mentioned this issue May 7, 2023

chore: update published build to python 3.11 vyperlang/vyper#3385

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`_struct.cpython-310-darwin.so` is not a fat binary #439

`_struct.cpython-310-darwin.so` is not a fat binary #439

charles-cooper commented Jun 18, 2022

bwoodsend commented Jun 18, 2022

panticmilos commented Jun 19, 2022

charles-cooper commented Aug 23, 2022

panticmilos commented Sep 14, 2022

charles-cooper commented Sep 14, 2022

IvanZosimov commented Sep 15, 2022

_struct.cpython-310-darwin.so is not a fat binary #439

_struct.cpython-310-darwin.so is not a fat binary #439

Comments

charles-cooper commented Jun 18, 2022

bwoodsend commented Jun 18, 2022

panticmilos commented Jun 19, 2022

charles-cooper commented Aug 23, 2022

panticmilos commented Sep 14, 2022

charles-cooper commented Sep 14, 2022

IvanZosimov commented Sep 15, 2022

`_struct.cpython-310-darwin.so` is not a fat binary #439

`_struct.cpython-310-darwin.so` is not a fat binary #439