New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Track a more recent upstream version of CPython that supports the Apple M1 platform? #36
Comments
I'm not planning on rebasing on to 3.10 soon, since rebasing is a lot of work, but I'd like to support Apple's M1 cpus. Python 3.9.1+ has native support for Apple silicon -- I'm not sure why it doesn't work. I'll look into it more in the next few weeks when I get access to an M1 laptop. |
BTW thanks for leading the nogil Python effort. As a maintainer of loky / joblib and scikit-learn, I really think the nogil project is very important for the future of the Python / data / ML ecosystem. loky is the default backend for multi-processing for joblib and therefore scikit-learn and we spent countless hours trying to make it work efficiently and robustly and its maintenance if far from free. All of this complexity would go away if we could efficiently use threads instead of sub-processes and having to deal with all kind of workarounds to try to limit memory usage and the overheads and brittleness inter-process communication via pickling. |
@ogrisel If you maintain loky, we should talk as we're looking to replace billiard with something that is easier to understand and debug. Drop me an email :) |
@ogrisel @colesbury Using an M1 MacBook Pro, I can successfully run
Update
|
Thanks @anna-hope. I did a |
For information, I am now running macOS 12.2.1 and the compiler env var evals to:
|
I think the Already fixed by mimalloc >= 1.7.2
and
after adopting patch (microsoft/mimalloc#346) from mimalloc, I got another error:
and
|
Thanks for the bug report and help with finding the underlying issues. I've pushed a branch "nogil-3.9.10" (https://github.com/colesbury/nogil/tree/nogil-3.9.10) that works on the Apple M1. It still needs some more testing, but I plan to eventually change the default "nogil" branch to point to it. For now, the "nogil-3.9.10" is unstable in the sense that I may push changes that don't preserve history. The significant changes are:
Please try it out if you can and let me know if you run into any problems. |
Looks like this version does not have cffi package for it, I am getting With this in pyproject.toml
|
This is fixed now in the
in order to get the changes in the nogil branch. Note that the fixed version of nogil Python should report as "Python 3.9.10". |
I would like to test this experimental branch on a regular basis on my Apple M1 dev laptop when working on the Scientific Python / PyData related projects so as to be able to report problems or performance improvements encountered with practical application cases.
However the build setup in this branch is a bit too old and the configure refuses to generate the Makefile for this platform:
Would it be possible to sync with branch with a more recent version of CPython (e.g. 3.10)?
EDIT: the error above was probably observed with a version of clang installed from conda-forge. I get a different error at build time when using the system compiler instead, see the discussion below: #36 (comment)
The text was updated successfully, but these errors were encountered: