-
Notifications
You must be signed in to change notification settings - Fork 921
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building a conda(-forge) package for this. #14
Comments
It currently fails with
Full build log at https://gist.github.com/xhochy/3290e73eba3fce442f4ba453fe8def60 As I have no experience with metal, I'm unsure about the right fix for this. |
Looking at your build log, it might be the case that you on a MacOS version that is not supported |
CMake correctly reports |
The build passes but sadly the import fails with
The issue is that
|
I have a PR ready that builds it for Apple Silicon on conda-forge: conda-forge/mlx-feedstock#3 Sadly this fails in cross-compilation mode. As we don't have Apple Silicon runners on conda-forge, this makes it hard to build. Locally (without cross-compilation), it passes. |
Cool! Would love to support install with Conda. Could you say more about how the distribution works? Right now we build w/ cross compilation for PyPI. Could we reuse that same machinery for Conda forge? |
It works similar to the PyPI machinery, but the main difference is that conda-forge brings its own compiler infrastructure. With Metal, this should be too much of a difference as it simply calls out into the SDK. The current problem is though that compilation stops with:
If I compile that natively (on Apple Silicon), I get the same error if I use "11.0" as the Xcode SDK and Deployment target. If I increase this then to 13.3, it passes. In the cross-compilation setup, the error sadly persists even with the more modern SDK and deployment target. |
One issue could also be the 13.3 SDK as only 13.4 is supported. Sadly, I don't know of a reliable way to retrieve a newer SDK in CI. |
Newer SDKs are usually available readily in the newer vm images we could use. |
Should be resolved in conda-forge/mlx-feedstock#4 |
@awni, we are again facing this error on arm64. Build passes, but runtime error... |
I've never seen that issue before (not being able to load the default device). You are compiling natively on an Apple silicon machine? Can you say more about the setup / environment. |
It happens both in cross- and non-cross-compilation mode. You can see the environment in the CI logs of conda-forge/mlx-feedstock#4 . This is a cross-compilation log but as the error is equal on whether cross-compiled or not, it should give you the relevant information. Happy to provide you with the relevant information, but I'm not sure what you would need. |
The only key difference (I can think of) is that we are using the LLVM Clang 16.0.6 (not the default Apple Clang 15.0.0 found on MacOS nowadays, at least on mine). We can potentially test with the Apple Clang provided in the images, but that may require us some additional work... |
I tried building with Clang 17.0.6 and did not encounter this issue:
I see you are continuing to work on this!! How is it going, any updates? |
From conda-forge/mlx-feedstock#4 (comment):
Has anyone tested cross-compiling here? I can test with Clang 17 we have available in conda-forge (update: tested, fails ❌) |
Hi I wonder if you try again on main it might find the device now? We changed the way we find the default device in #370 |
@ngam could you say more about packaging the library correctly? It seems to work fine with PyPi? I tested it here: https://test.pypi.org/project/awni-test-mlx/0.0.7/ |
Oh... I only tested for conda-forge (not pypi) and assumed it would be the same for pypi. This may be limited to conda packaging; and if so, @xhochy or I would submit a patch if needed. For now, I believe we have this working on conda-forge (pending a new release from you and pending @xhochy's review). See conda-forge/mlx-feedstock#4 (comment). |
Oh ok, let me know if we need to change anything on our end! |
@ngam it turns out your were 100% right, the packaging was broken (🤦 I didn't consider that I had gguf already installed on the machines I tested on). We fixed this in the latest release. Hopefully that makes it easier to get the conda package working! |
0.0.9 builds and passes. We should be able to release mlx on conda-forge soon 🎉 |
…and pushed to conda-forge. Verified that it works. I successfully fine-tuned |
This is so awesome! One more dumb question: as we do new releases, how does the conda forge distribution get updated? |
conda-forge has a bot called @regro-cf-autotick-bot that will issue new PRs to https://github.com/conda-forge/mlx-feedstock where I will review and merge (and I have hopes that @ngam will continue to support me there). Once the PR is merged, the package will be available roughly 1h later on conda-forge. |
Thanks! And thanks for setting this up. I will plan to add the new install path to our docs |
Thank you all for working on this! Very happy to see it landing on conda-forge. |
Thanks @xhochy and @awni. I think we will benefit from someone on the mlx side getting involved as well (anyone interested we can add them as maintainers on the conda-forge side). As always @awni, don't hesitate to ping one of us if needed We are compiling with our own LLVM clang compilers and we are building against macos sdk 13.3. If at any point, we need to increase the sdk version, e.g., for new features in metal, we can do that |
It would also be nice to have a binary for this on conda-forge. You can see my start at implementing a recipe for this at https://github.com/conda-forge/staged-recipes/pull/24687/files As the CI over there doesn't build for Apple silicon, we don't see any failure. Thus I would use this issue to reach out for help with build issues.
The text was updated successfully, but these errors were encountered: