-
Notifications
You must be signed in to change notification settings - Fork 21.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initializing libiomp5.dylib, but found libomp.dylib already initialized. #78490
Comments
I've ended "fixing" the issue by wiping my conda setup, moving to py3.10 and reinstalling dependencies. The issue was definitely not related to TorchVision as I managed to reproduce it only using Core. It could be related to some weird dependency on my system though. As discussed with @atalman I'll leave the issue open just in case this is something suspicious that needs to be investigated on the future. I know there were a bunch of macOS changes recently on OK. Feel free to close or move to Core. |
@datumbox you should have run |
Looks like its conda issue, I was able to reproduce this: conda install pytorch torchvision torchaudio -c pytorch-nightly
|
@atalman can you please run |
Here they are, this config was failing
|
Looks like I was able to mitigate this issue by doing following
So looks like problem could be related to openmp
|
Ok, can we please clarify the following: what conda package provides libiomp5.dylib? If it's provided by MKL base, that we do not need to bundle it with the package, but rather take it from conda, i.e. following function should not be called when building conda packages for MacOS Line 432 in 18d46ea
|
After doing some more research I found out that this error is resolved by
however we can reintroduce this issue by installing
which installs following packages:
|
Summary: Fixes #78490 Following command: ``` conda install pytorch torchvision torchaudio -c pytorch-nightly ``` Installs libiomp . Hence we don't want to package libiomp with conda installs. However, we still keep it for libtorch and wheels. Pull Request resolved: #78632 Approved by: https://github.com/malfet Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/ca7f948806d296727d16233dd82392fa6664aa89 Reviewed By: b0noI Differential Revision: D36854265 Pulled By: atalman fbshipit-source-id: 1b9a2f034cac822d9936febaa7b94213c31af19f
I am also getting this with python 3.10 on intel mac and the released pytorch 1.12.0
|
@jamt9000 please use workaround this issue by setting |
@atalman Thanks for jumping in to fix the issue. I don't see a PR to update TorchVision to point at a different binary. Do we need any action on our side or all changes are done on Core? cc @YosuaMichael |
reopeing, cherry-pick required for 1.12.1 |
closing for now as it is resolved in master, will be cherry picking it for 1.12.1 |
…torch#78632) Summary: Fixes pytorch#78490 Following command: ``` conda install pytorch torchvision torchaudio -c pytorch-nightly ``` Installs libiomp . Hence we don't want to package libiomp with conda installs. However, we still keep it for libtorch and wheels. Pull Request resolved: pytorch#78632 Approved by: https://github.com/malfet Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/ca7f948806d296727d16233dd82392fa6664aa89 Reviewed By: b0noI Differential Revision: D36854265 Pulled By: atalman fbshipit-source-id: 1b9a2f034cac822d9936febaa7b94213c31af19f
…81873) Summary: Fixes #78490 Following command: ``` conda install pytorch torchvision torchaudio -c pytorch-nightly ``` Installs libiomp . Hence we don't want to package libiomp with conda installs. However, we still keep it for libtorch and wheels. Pull Request resolved: #78632 Approved by: https://github.com/malfet Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/ca7f948806d296727d16233dd82392fa6664aa89 Reviewed By: b0noI Differential Revision: D36854265 Pulled By: atalman fbshipit-source-id: 1b9a2f034cac822d9936febaa7b94213c31af19f
I'm still getting this problem with 1.12.1. The workaround Edit: I did some debugging, and if I import |
I have the same issue as @samgelman on my MacOS. Notably 1.10.1, 1.11.0 works fine but 1.10.2 and 1.12.1 shows:
One observation is that both 1.10.1 and 1.11.0 also come with The workaround Below is a working env:
|
same issue here... after spending several days fixing this issue, I'm actually quitting doing ML on M1 mac |
If I install pytorch first in my conda environment, I see the actual files libiomp5.dylib in two places.
I do not get the OpenMP error. However, if I install jupyter into this environment the file libiomp5.dylib gets updated to a link pointing to libomp.dylib. After which, the error comes back again. Is there a reason that libiomp5.dylib is present in two locations after pytorch install ? |
On my MacBookPro (intel cpu), I got the similar error:
Add the env var, get a segment fault error:
I search for no many
|
Having the same issue on Pytorch 2 here:
The line |
I am using this on windows11 and getting the same bug. None of the suggested solutions works except for setting the environment variable. |
This issue still exists for macOS using Rosetta2 (native arm64 works). Haven't found a working solution yet except the env variable. |
@FreeBlues do you find the solution? |
I tried as the xgboost issue suggested, then solved:
|
I'm getting this too when trying to run a In [1]: import torch
In [2]: x = torch.rand(1024 * 4, 1024 * 4)
In [3]: @torch.compile(fullgraph=True)
...: def foo2(x):
...: return torch.sin(x) * torch.cos(x)
...:
In [4]: foo2(x) I get this error:
Okay, I run
Looks like I went from having no OpenMP at all to having too much of it?
|
I installed the |
🐛 Describe the bug
The issue appears on MacOS py3.8, it started after updating to the latest nightly
1.13.0.dev20220525-py3.8_0
from core (previously I was at1.12.0.dev20220309-py3.8_0
, so the issue could have been introduced earlier than May 25th). I'm receiving the following after importing numpy and pytorch together:Strangely, importing first torch works:
Setting
KMP_DUPLICATE_LIB_OK=TRUE
as env var solves the issue while invoking from console:Sometimes I get segfaults thought, this doesn't seem like a stable solution.
Versions
Latest Core nightly (20220525).
The text was updated successfully, but these errors were encountered: