llama2 Segmentation fault (core dumped) on i7-9700K

[JJJohnathan]

The results of running https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/GPU/PyTorch-Models/Model/llama2 are as follow:

ipex-llm/python/llm/example/GPU/PyTorch-Models/Model/llama2$ python ./generate.py --repo-id-or-model-path ~/Downloads/model
/home/openvino-test-01/miniforge3/envs/ipex_llm/lib/python3.11/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
2024-06-14 12:12:23,960 - INFO - intel_extension_for_pytorch auto imported
Loading checkpoint shards: 100%|████████████████| 2/2 [00:00<00:00, 6.37it/s]
2024-06-14 12:12:24,393 - INFO - Converting the current model to sym_int4 format......

LIBXSMM_VERSION: main_stable-1.17-3651 (25693763)
LIBXSMM_TARGET: hsw [Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz]
Registry and code: 13 MB
Command: python ./generate.py --repo-id-or-model-path /home/openvino-test-01/Downloads/model
Uptime: 31.656327 s
Segmentation fault (core dumped)

[Oscilloscope98]

Hi @JJJohnathan,

Would you mind having a check on your glibc version? We recommend glibc with version >= 2.28 :)

Besides, it would be helpful to provide more information regarding your environment through check_env.sh.

[JJJohnathan]

@Oscilloscope98 OK, The result of running check_env.sh is as following:

---

PYTHON_VERSION=3.11.9

transformers=4.36.2

torch=2.1.0a0+cxx11.abi

ipex-llm Version: 2.1.0b20240613

ipex=2.1.10+xpu

CPU Information:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 39 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz
CPU family: 6
Model: 158
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
Stepping: 13
CPU max MHz: 4900.0000
CPU min MHz: 800.0000
BogoMIPS: 7200.00

Total CPU Memory: 31.1971 GB
Memory Type: DDR4

Operating System:
Ubuntu 22.04.4 LTS \n \l

---

Linux openvinotest01-Z390-AORUS-ULTRA 6.5.0-35-generic #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

CLI:
Version: 1.2.35.20240425
Build ID: 00000000

Service:
Version: 1.2.35.20240425
Build ID: 00000000
Level Zero Version: 1.16.0

Driver Version 2023.16.12.0.12_195853.xmain-hotfix
Driver Version 2023.16.12.0.12_195853.xmain-hotfix

Driver related package version:
ii intel-fw-gpu 2024.17.5-32922.04 all Firmware package for Intel integrated and discrete GPUs
ii intel-i915-dkms 1.24.2.17.240301.20+i29-1 all Out of tree i915 driver.
ii intel-level-zero-gpu 1.3.29138.29-88122.04 amd64 Intel(R) Graphics Compute Runtime for oneAPI Level Zero.

igpu not detected

xpu-smi is properly installed.

No device discovered
GPU0 Memory size=16M

00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02) (prog-if 00 [VGA controller])
DeviceName: Onboard - Video
Subsystem: Gigabyte Technology Co., Ltd CoffeeLake-S GT2 [UHD Graphics 630]
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at 50000000 (64-bit, non-prefetchable) [size=16M]
Memory at 40000000 (64-bit, prefetchable) [size=256M]
I/O ports at 3000 [size=64]
Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
Capabilities:

[JJJohnathan]

ldd (Ubuntu GLIBC 2.35-0ubuntu3.8) 2.35
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

[Oscilloscope98]

Hi @JJJohnathan,

We haven't conducted verification for IPEX-LLM on the iGPU of 9th Intel Core yet. For Intel iGPU, we have mainly conducted verification for the ones of Intel Core above or equal to 11th.

For Llama2 text generation example on i7-9700K, you may have a try on CPU first through this example: https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels/Model/llama2 . Please note that this example will require creating a environment for CPU :)