-
Notifications
You must be signed in to change notification settings - Fork 236
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Drivers from 4885 and newer break IPEX for native windows. #442
Comments
+1.
Under some circumstances, I can see "Abort was called at 198 line in file:" -- I believe this is raised from compute runtime. I'm trying to isolate the issue. |
Just to confirm, I also got this a few times. |
accelerate with --use_xpu or ipex enabled in config also throws |
It seems that driver 4885 was breaking backward compatibility against previous drivers. The officially released IPEX Windows JIT wheels work fine with the following reproducer (the image was generated as expected): import torch
import intel_extension_for_pytorch as ipex
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16).to("xpu")
prompt = "a photograph of an astronaut riding a horse"
image = pipe(prompt).images[0]
image.save(f"astronaut_rides_horse.png") However, if I use IPEX AOT wheels built from source with driver 4676 (or earlier) (for example, https://github.com/Nuullll/intel-extension-for-pytorch/releases/tag/v2.0.110%2Bxpu-master%2Bdll-bundle), the program crashes. pip install https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master%2Bdll-bundle/torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl
pip install https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master%2Bdll-bundle/intel_extension_for_pytorch-2.0.110+gitc6ea20b-cp310-cp310-win_amd64.whl
pip install diffusers transformers
set SYCL_PI_TRACE=2
python reproducer.py 1> trace.log 2>&1 ---> piKernelCreate(
<unknown> : 000001CA0D158570
<const char *>: _ZTSZZN2at15AtenIpexTypeXPUL20launch_legacy_kernelIZNS0_18dpcpp_loops_kernelIZZZNS_4impl21copy_device_to_deviceERNS_14TensorIteratorEbENKUlvE3_clEvENKUlvE9_clEvEUlN3c104HalfEE_Lb0ELb1EEEvRNS_18TensorIteratorBaseET_EUliE_EEvxRKSD_ENKUlRN4sycl3_V17handlerEE_clESK_EUlNSI_7nd_itemILi1EEEE_
<unknown> : 0000000DD9BE8C78
PI ---> (*RetKernel)->initialize()
PI ---> piProgramRetain(Program)
) ---> pi_result : PI_SUCCESS
[out]<unknown> ** : 0000000DD9BE8C78[ 000001C9B5B0E8D0 ... ]
...
---> piEnqueueKernelLaunch(
<unknown> : 000001C8A3717830
<unknown> : 000001C9B5B0E8D0
<unknown> : 1
<unknown> : 0000000DD9BEA658
<unknown> : 0000000DD9BEA628
<unknown> : 0000000DD9BEA640
<unknown> : 0
pi_event * : 0000000000000000[ nullptr ]
pi_event * : 000001C8A7A7D1D8[ 0000000000000000 ... ]
PI ---> Queue->insertStartBarrierIfDiscardEventsMode(CommandList)
PI ---> EventCreate(Queue->Context, Queue, ForceHostVisible, Event)
PI ---> piEventRetain(*Event)
PI ---> piKernelRetain(Kernel) Crashed while executing Probably I should compile IPEX with driver 4885? |
You could try and see, but the official wheels haven't been updated (afaik) so I don't think they were compiled on the latest drivers. Maybe the new drivers break something in AOT? |
I tried compiling IPEX AOT for Arc with driver 4887. The reproducer still crashes with the same SYCL PI TRACE log. |
Are there any updates on whats going on with the newest drivers? I personally haven't tried the very latest but have heard it is also not working from others.(I may give it a shot if someone says otherwise). Any progress on figuring out what's happening? |
I can confirm that Driver 4885, 4887 and 4900 all cannot work with IPEX AOT, simply because they ship the same Level Zero Compute Runtime "1.3.27193". |
I take it this is completely driver level and no way to override and install the older runtime version? |
I tried to replace the driver storage files |
that's way above my level, however in my folder I do not see a ze_intel_gpu64.dll in the main folder but only in one of the driver state repository folders, this is driver 4676 |
Yes, correct. 4 |
The issue is gone with Driver 4952 |
Describe the issue
I have tried running it in both Sd.Next and ComfyUI and both fail when trying to generate an image. There is no error message it just seems to crash the Webui comletely. 4676 and older worked perfectly fine. Since there is no error message I can't really tell you what is broken. I believe the driver team is notified but I'm not sure what they can do since it's not officially supported, so I figured I would also post in here as well.
Wsl2 seems to still work fine.
a750
Windows 11
AOT compiled IPEX for windows
ryzen 5600
32gb of ddr4 at 3200
The text was updated successfully, but these errors were encountered: