Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort out the issue of bundling libraries for different mac architectures #2409

Closed
aivarannamaa opened this issue Aug 28, 2022 · 4 comments
Closed

Comments

@aivarannamaa
Copy link
Member

The XXL build for Mac is more difficult now, as I need to account for both Arm and Intel targets. The bundled Python itself is universal2 build, but most packages publish separate wheels for intel and arm and when I pip-install the packages into the bundle on my Intel Mac, only Intel wheels get downloaded.

I heard that it's possible to make pip ignore published binaries and build universal2 wheels during the install. If I manage to do this, then I'll restore the XXL bundle. Or I'll start publishing separate installers for Intel and Arm.

@aivarannamaa aivarannamaa added this to the 4.0.1 milestone Aug 28, 2022
@aivarannamaa
Copy link
Member Author

aivarannamaa commented Aug 29, 2022

I had forgotten, that the issue of different architectures also touches the regular bundle -- certain dependencies of paramiko (used for remote interpreters) are available only as arm or intel wheels, not as universal2.

The simplest solution would be to start publishing separate Thonny installers for Intel and Arm Macs (even though bundled Python itself is universal2). This could pose some difficulties for defining Thonny's homebrew cask, though.

@andypiper, you just made the PR for Thonny's cask. Do you think you can handle separate downloads for Intel and Arm for Thonny 4.0.1? I found this, which gives me the impression that such a thing should be possible: https://github.com/Homebrew/brew/pull/10683/files

I also considered bundling both arm and intel binaries for packages without universal2 builds and having the installer sort it out which files to install, but this solution would be much more complex to implement and it would increase the installer size.

@aivarannamaa aivarannamaa changed the title Sort out the XXL bundle for mac Sort out the issue of different mac architectures Aug 29, 2022
@aivarannamaa aivarannamaa changed the title Sort out the issue of different mac architectures Sort out the issue of bundling libraries for different mac architectures Aug 29, 2022
@aivarannamaa
Copy link
Member Author

I also considered bundling both arm and intel binaries for packages without universal2 builds and having the installer sort it out which files to install, but this solution would be much more complex to implement and it would increase the installer size.

I could use this approach if I ditched the XXL build for Mac, as the regular build has only couple of problematic dependencies and it's not too hard to update the postinstall script to overwrite a few specific files. But this approach wouldn't scale to XXL bundle.

@andypiper
Copy link

I can look into the preferred options for ARM vs Intel brew cask installs, sure. Thanks for tagging me in 👍🏻

@aivarannamaa
Copy link
Member Author

I solved the issue for the regular installer. I replaced cryptography so-files with universal2 ones and made the installer swap out so-file for cffi in the postinstall script when run on Arm. This means, I can continue publishing single installer for both Intel and Arm macs.

I decided to skip publishing the XXL build for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants