Segmentation fault after change cuda-based python code to xpu-based python code with codegeex2-6b mode

[ganghe]

Hi Team,

I tried to change codegeex2 cuda-based python example code to xpu-based python code with codegeex2-6b mode file,
but the program always crashed.

The cuda-based python file is here,
https://github.com/THUDM/CodeGeeX2/blob/main/demo/fastapicpu.py

I can run it with cpu device, e.g. "python fastapicpu.py --model-path codegeex2-6b --cpu".
but, I changed cuda-based python code to xpu-based python code, run it with gpu device
with the command as below
source /opt/intel/oneapi/setvars.sh --force
python -X faulthandler fastapicpu.py --model-path codegeex2-6b

Then, the program crashed at line 207,
"model = model.to('xpu')"

The related files are attached.
fastapicpu.old.txt - cuda based python file
fastapicpu.py.txt - xpu based python file
cmd.txt - startup command script
output.txt - program output log
mode file - https://hf-mirror.com/THUDM/codegeex2-6b

cmd.txt
fastapicpu.old.txt
fastapicpu.py.txt
output.txt

[NovTi]

Thank you for your question. This problem has been reproduced and I am currently working on it

[NovTi]

This issue is caused by the module import order and we suggest you put import torch above from transformers import AutoTokenizer's and from bigdl.llm.transformers import AutoModel like this to solve this error.

import torch
from transformers import AutoTokenizer
from bigdl.llm.transformers import AutoModel

[ganghe]

Hi NovTi,

Thank for your helps.
The fix works for me.

-Gang