Any chance for multi gpu support for Intel Arc?

[johnnnymachine]

Hi, as the title suggest, I'm getting segfault when trying to run llama with 2x Intel Arc a750's.. This seems to be a limitation with the SYCL backend.. works fine with the Vulkan backend.. Its possible to get running in single gpu mode using 'export ONEAPI_DEVICE_SELECTOR="opencl:1"'

[NeoZhangJianyu]

@johnnnymachine
SYCL backend support multiple GPUs in same machine.

[johnnnymachine]

Hi,

This is my output from GDB on linux debian 13, when i try run in dual gpu mode:

Thread 1 "llama-cli" received signal SIGSEGV, Segmentation fault.
0x00007fffd15cd04a in NEO::StagingBufferManager::isValidForCopy(NEO::Device const&, void*, void const*, unsigned long, bool, unsigned int) ()
from /usr/local/lib/intel-opencl/libigdrcl.so

Any chance of some help with this? Would be nice to get it working..
Johnny

[NeoZhangJianyu]

@johnnnymachine

export ONEAPI_DEVICE_SELECTOR="opencl:1" is not correct for SYCL backend.
Use export ONEAPI_DEVICE_SELECTOR="level_zero:1".

Could you share the full log?

I test with the latest code. It supports two Flex 170 GPUs successfully.
Please note, the multiple GPUs should be same mode.

Here is an example:
./examples/sycl/run-llama2.sh support only one GPU as default.
To support multiple GPUs, please modify ./examples/sycl/run-llama2.sh as:

export ONEAPI_DEVICE_SELECTOR="level_zero:0"
->
#export ONEAPI_DEVICE_SELECTOR="level_zero:0"

Here is the part of log of my case:

Found 2 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|         Intel Data Center GPU Flex 170|  12.55|    512|    1024|   32| 16225M|         1.6.33578+15|
| 1| [level_zero:gpu:1]|         Intel Data Center GPU Flex 170|  12.55|    512|    1024|   32| 16225M|         1.6.33578+15|
SYCL Optimization Feature:
|ID|        Device Type|Reorder|
|--|-------------------|-------|
| 0| [level_zero:gpu:0]|      Y|
| 1| [level_zero:gpu:1]|      Y|
llama_context:  SYCL_Host  output buffer size =     0.12 MiB
llama_kv_cache:      SYCL0 KV buffer size =  1088.00 MiB
llama_kv_cache:      SYCL1 KV buffer size =   960.00 MiB
llama_kv_cache: size = 2048.00 MiB (  4096 cells,  32 layers,  1/1 seqs), K (f16): 1024.00 MiB, V (f16): 1024.00 MiB

[johnnnymachine]

Hi,

I tried it again, and for some reason its working now? I tried unsetting ONEAPI_DEVICE_SELECTOR and also tried setting to level_zero:* But thanks for your help!

Johnny