New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Documentation clarification for installing MKL on Mac OS X #31
Comments
Added to the docs. Thanks. |
@JulioBarros Thanks, I think there is a typo in the first command which should have the forward slash / before *
|
Hi, I just want to comment on that linking
does not work for me on Mojave with MKL 2019.5.281. I have to link:
Got the idea from this post (second problem). |
Couldn't get the above tricks to work anymore (on Mac OS Catalina 10.15.4, and Intel MKL 2020.1, and oracle-jdk installed through brew cask). Everything worked fine if I disabled SIP, so that DYLD_LIBRARY_PATH became visible to the repl, but nothing else seemed to work: had the relevant dylib files in usr/local/bin, had the files in /Library/Java/Extensions, etc, to no effect. Not really satisfactory, but spend more time than I wanted on trying to get this to work. One thing to get things to work with SIP disabled after installing the Intel MKL is to add these three lines to your .zshenv:
That will set several variables, including DYLD_LIBRARY_PATH. edit - should have said I tried "usr/local/lib" - was mucking around with the java binary later on, and confused "usr/local/lib" and "usr/local/bin" in the writeup. |
Thank you. Important: dylib files should be in |
On MacOS 10.5.5 I also had to run
|
Worked on Mac 10.14.6 ...
|
Since 0.35.0 there is also the zero-install option: https://dragan.rocks/articles/20/Clojure-Neanderthal-MKL-acceleration-out-of-the-box |
Looks great, but I might still be missing a step somewhere. Downloaded hello-world and tried to run it, and I get
|
Do you use 0.35.0, and do you have this dependency enabled? ;;Optional. If bytedeco is not present, a system-wide MKL is used. BTW. Can you please run lein repl in terminal, and paste the complete output, so I can try to see why it fails on your machine? |
I could not get MKL jar to work on my Mac so I installed the full MLK as described above successfully. If I now simply add the jar dependency back...
I get the following error in my REPL when I load a namespace requiring
Before I fully installed MLK and only added the jar dependency I got an error referencing the /private directory cache on my machine. |
I only have access to an old macOS from 5 years ago, where everything works. Later Apple started throwing wrenches in the wheels, so each new macOS disables something. Any info with specific versions and combinations is appreciated. Unfortunately, I can't help much in straightening it completely. |
Hi Dragan, Thanks for all your efforts - got it to work straight from the start on Windows 10, but I tried both on my own macbook air 2013 (MacOSX 10.15.6), and my gf's mac (idem, but with macOS 10.14.6, and as she doesn't muck about with clojure or MKLs, it was a clean mac as far as clojure/intel_MKL was concerned), and got the same error. Here is on my mac:
Java version:
the project.clj:
And if I do a sudo find * | grep mkl, I see that mkl is located in the places you might expect it:
|
I have the same problem on Mac OS X 10.14.5 running Java 13.0.1. Thanks for your help!
user=> (require '[uncomplicate.neanderthal.native :refer [dv dge]]) user=> *e |
@genovese Does it work when you remove the bytedeco dependency and just use the system-wide libraries? In case you wonder where to find these libraries, you can even rename the bytedeco jar to zip, unzip it, and then use the dylib files from there. You don't need to fully install MKL, you just need to add it to the DYLD_LIBRARY_PATH (or whatever is current for your particular version of macOS). Note: you'd need to configure SIP and whatever developer blocking mechanisms Apple activated in addition to it. |
I did not yet install the library system wide because I expected the bytedeco package to handle it. I will now, however, and will report back. Thanks. |
Bytedeco is under the same macOS restrictions as any other means of adding native libraries. Maybe the problem is that you didn't disable Apple's restrictions (SIP, etc.). Can you please try just bytedeco with SIP enabled, before installing MKL system-wide? |
I already have full install of MKL working. As an experiment as suggested by blueberry I ...
Just out of curiosity I also tried executing
I will go back to full install of MKL that worked using steps I mentioned earlier in this thread. NOTE: I am using Java 11 and have uncommented line |
I've pushed new MKL loading implementation to github. Can you please clone the neanderthal repo, do only (Of course, both ways work on my machines, but I need someone of you to try whether it works on yours too). |
Yes, I'll do that. Thanks
…On Sat, Aug 22, 2020 at 4:27 AM Dragan Djuric ***@***.***> wrote:
Bytedeco is under the same macOS restrictions as any other means of adding
native libraries. Maybe the problem is that you didn't disabled Apple's
restrictions (SIP, etc.).
Can you please try just bytedeco with SIP enabled, before installing MKL
system-wide?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#31 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABQMIOP7BPK4I6AVOO5EP3SB56PRANCNFSM4D2VBKOA>
.
|
blueberry do we need dependency uncomplicate:commons:jar:0.12.0-SNAPSHOT for uncomplicate:neanderthal:jar:0.36.0-SNAPSHOT? |
Yes. |
I lein installed both
I will try again later when I have time. |
If you can, please see the mkl.clj file, an the way I load the namespace. If possible, try to experiment with various ways of blocking the namespace loading until mkl_rt has been loaded. For example, |
I can confirm that a fresh install of MKL on a different laptop works if the contents of both |
Unfortunately I have to disagree with the above. I couldn't get Neanderthal to work with MKL on my MacOS with none of the suggestions in this issue. I always get either |
same here |
Are you sure you've disabled SIP? |
Ah no that I haven't tried to be honest. Do I understand correctly that Neanderthal as of today cannot be used without turning off SIP on MacOS? |
I don't know since I don't have a recent mac. Some users say it can, some say it can't. It probably can, but you have to know macOS and Apple's countless roadblocks well enough to control them. |
I see. Is there a Neanderthal docker image available? Sorry if off topic but maybe it is relevant for Mac users who can't get MKL to behave.. |
(I'm using Mojave and do not have SIP disabled) |
I also have Neanderthal working on another machine with Catalina without SIP disabled - same setup as above - can only make it work using JDK 10.0.2 I don't think SIP is a factor here |
Oh, wait a minute here. I am definitely positive that SIP is absolutely central to this issue. The only thing that is debatable is whether there is a way to configure macos path without SIP. In your case it worked, but only with Java 10.0.2 (which is strange). IMHO, for newbies, it is best to disable SIP and make MKL work first, and then, perhaps, explore how to make this configurable while re-enabling SIP. (On my machine, I had to disable SIP, but I also need to use MKL in the compilation step, while the users don't need to compile anything, just make MKL dylibs visible to the JVM) |
For whoever did not manage to install Neanderthal on MacOS, I used this docker image: https://github.com/ComputeSoftware/java-neanderthal-docker/blob/master/tools-deps/Dockerfile , to which I added:
Build the image, run it with port forwarding 8888 and connect with CIDER. Half an hour well spent. |
Mac Catalina 10.15.6 having the same problem
Disabling SIP is not an option for me because of a bug with Mac recovery mode On the other hand I don't think is SIP, cause I actually tried the hello-world example from this repo and in that case the import worked
I examined the repo where the error repro but I couldn't get what's the difference. I am really happy @lccambiaghi made a docker image! That really saves my day |
@dragandj would you be interested in a PR with the Dockerfile and instructions on how to build, run and connect to be added to the README? |
@lccambiaghi Sure, if you (or someone else) would accept the responsibility to keep it up to date with latest version of Neanderthal and answer potential questions, since I don't use docker nor have appropriate macOS to test it. |
I added #101 . |
I stumbled around trying to install MKL and figured out setting DYLD_LIBRARY_PATH on MAC OS X doesn’t work as one might expect anymore because because of system integrity protection (SIP) http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/
I instead created links from the MKL libs to an existing allowed directory and that seems to work. For example:
ln -s /opt/intel/lib* /usr/local/lib
ln -s /opt/intel/mkl/lib/* /usr/local/lib
This is mentioned the docs (thanks!) but wanted to point out that it may be necessary if you don't want to relax the SIP settings and I thought clearer/stronger suggestion would alert people that this is an option. Thanks again.
The text was updated successfully, but these errors were encountered: