Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImportError: /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent #10550

Open
rmurtazi opened this issue Mar 26, 2024 · 15 comments
Open

ImportError: /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent #10550

rmurtazi opened this issue Mar 26, 2024 · 15 comments
Assignees
@qiuxin2012
Labels
user issue

Comments

@rmurtazi
Copy link

I tried to install IPEX-LLM using the following instruction: https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html

Installation works w/o any problems. When I try to do validation of installed packages I get an error:

(llm) gta@DUT7213ATSM:~$ python
Python 3.9.19 (main, Mar 21 2024, 17:11:28)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

from ipex_llm.transformers import AutoModel, AutoModelForCausalLM
Traceback (most recent call last):
File "", line 1, in
File "/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/ipex_llm/init.py", line 23, in
from .optimize import optimize_model
File "/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/ipex_llm/optimize.py", line 17, in
import torch
File "/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/init.py", line 235, in
from torch._C import * # noqa: F403
ImportError: /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent
@yangw1234
Copy link
Contributor

yangw1234 commented Mar 26, 2024 •
edited

Hi @rmurtazi , could you run this script in your python environment and share the result? https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/scripts/env-check.sh

@tsantra
Copy link

tsantra commented Mar 26, 2024

Hi yangw1234 , I am also getting this same error.

image

@tsantra
Copy link

tsantra commented Mar 26, 2024 •
edited

log.txt
@yangw1234 attaching the log file which has the output of the env-check.sh script. It says IPEX is not installed.
I have followed the following installation steps:

image

@yangw1234
Copy link
Contributor

@tsantra could you try pip list | grep torch?

@tsantra
Copy link

tsantra commented Mar 26, 2024

@yangw1234
image

@yangw1234
Copy link
Contributor

@tsantra could you try pip list | grep torch?

Hi @tsantra, I see your oneapi version is 2024.1 and current intel ipex/pytorch is built on openapi 2024.0, so I think this is the problem.

Intel just released 2024.1 recently so the installation instructions needs to be updated because it will install the latest oneapi version by default.

@tsantra
Copy link

tsantra commented Mar 26, 2024

@yangw1234 could you please give me the instruction to downgrade to 2024.0

@qiuxin2012 qiuxin2012 self-assigned this Mar 27, 2024
@qiuxin2012
Copy link
Contributor

qiuxin2012 commented Mar 27, 2024 •
edited

If you install oneapi using apt, you can:

sudo apt autoremove -y intel-basekit
sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

If you install oneapi using offline package, you can delete the 2024.1 version(just remove the oneapi folder), and install https://registrationcenter-download.intel.com/akdlm/IRC_NAS/163da6e4-56eb-4948-aba3-debcec61c064/l_BaseKit_p_2024.0.1.46_offline.sh

@rmurtazi
Copy link
Author

@yangw1234
After downgrading to oneAPI 2024.0.1 I get the following:
ython 3.9.19 (main, Mar 21 2024, 17:11:28)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

from ipex_llm.transformers import AutoModel, AutoModelForCausalLM
/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
2024-03-27 08:07:49,448 - INFO - intel_extension_for_pytorch auto imported

@qiuxin2012
Copy link
Contributor

Sorry I just find sudo apt install intel-basekit=2024.0.1-43 is installing 2024.1 again, you should use below command:

sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

@qiuxin2012
Copy link
Contributor

qiuxin2012 commented Mar 27, 2024 •
edited

@yangw1234 After downgrading to oneAPI 2024.0.1 I get the following: ython 3.9.19 (main, Mar 21 2024, 17:11:28) [GCC 11.2.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

from ipex_llm.transformers import AutoModel, AutoModelForCausalLM
/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
2024-03-27 08:07:49,448 - INFO - intel_extension_for_pytorch auto imported

It's just some warnings, you can ignore them. From the INFO, it looks you have imported IPEX correctly.

@jason-dai
Copy link
Contributor

Sorry I just find sudo apt install intel-basekit=2024.0.1-43 is installing 2024.1 again, you should use below command:

sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

@chtanch can you please verify?

@hkvision
Copy link
Contributor

Sorry I just find sudo apt install intel-basekit=2024.0.1-43 is installing 2024.1 again, you should use below command:

sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

@chtanch can you please verify?

@NovTi Will help fix it.

@NineMeowICT
Copy link

NineMeowICT commented Apr 12, 2024 •
edited

Downgrading oneAPI is a fallback though. If you want to use the latest oneAPI. You can try to run the following command after the normal installation:

python -m pip install torch==2.1.0.post0 torchvision==0.16.0.post0 torchaudio==2.1.0.post0 intel-extension-for-pytorch==2.1.20+xpu oneccl_bind_pt==2.1.200+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

I knew this method from another issue pytorch/pytorch#123097 describing a same problem.

ps: I searched this repo and found it seems that Intel hasn't updated its package requirements completely so ipex-llm will raise the error when using the new version of oneAPI. Come on, Intel, just fix it!

@John-D-Boom
Copy link

If you don't have sudo access, simply adding mkl==2024.0.0 to my environment.yml was enough to resolve the issue for now by downgrading mkl back to a compatible version.

@liu-shaojun liu-shaojun mentioned this issue Apr 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@qiuxin2012 qiuxin2012

Labels
user issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@jason-dai @qiuxin2012 @yangw1234 @hkvision @tsantra @John-D-Boom @rmurtazi @NineMeowICT