Steps to run SD.Next with Intel Arc GPU on native windows (IPEX) #2023
Replies: 8 comments 59 replies
-
|
Thanks! Btw, this should probably be a Wiki page? |
Beta Was this translation helpful? Give feedback.
-
|
Sure, will move it to wiki! |
Beta Was this translation helpful? Give feedback.
-
|
btw, ping me or @Disty0 to commit to wiki - i had to limit write access to collaborators only as there were some bad updates by anonymous users. |
Beta Was this translation helpful? Give feedback.
-
|
to tackle the tbb version probleme instead of opening any cmd shell you can open the terminal named "Intel oneAPI command prompt for Intel 64 for Visual Studio 2022", i think it is created when you install the VS integration part of OneApi . it will automaticaly source the setvars and apparently do a little more than that. Now i'm facing no error or warning in the sd.next log but it does take a awfull lot of time to load a model and more specificaly the embedding part of the checkpoint up to 100 secondes just for that. On wsl2 it only take a few seconds. After it ultimately load and is finally ready it won't start generating anything. it juste hang there writing nothing in the log file. here is the sdnext logs and the python packages installed. |
Beta Was this translation helpful? Give feedback.
-
Thanks for the info. I'll check your approach.
Yes, startup and first-time inference (whenever your generation parameters change) is dramatically slow for IPEX on native windows, because the Intel-released IPEX windows wheels require further JIT (Just-In-Time) compilation before the actual computation. The linux wheels are pre-built AOT (Ahead-Of-Time) binaries. See intel/intel-extension-for-pytorch#399 You might need to wait a little longer to see the first image output. For your reference, with Arc A770 + i9-13900:
|
Beta Was this translation helpful? Give feedback.
-
|
Good to know, I thought I was the only one with this issue. I'll stick to WSL2 for now. Thanks for your hardwork, keep it up |
Beta Was this translation helpful? Give feedback.
-
|
wow 6 mins to start. I never would have find out because i never would have waited that long without anything showing it's working. thank for the intel |
Beta Was this translation helpful? Give feedback.
-
Fun facts:
|
Beta Was this translation helpful? Give feedback.
-
|
Updated (rewrited) the tutorial. I think this would be a much simpler setup than the old one. |
Beta Was this translation helpful? Give feedback.
-
|
@kunumigab create an issue, don't comment on unrelated post (just because it talks about ipex). |
Beta Was this translation helpful? Give feedback.
-
|
@vladmandic what do you think about using @Nuullll's IPEX build instead of Intel's? |
Beta Was this translation helpful? Give feedback.
-
|
totally not against it |
Beta Was this translation helpful? Give feedback.
-
|
Added them in dev branch: 2f17cef |
Beta Was this translation helpful? Give feedback.
-
|
i guess the wiki should be simpler then :) |
Beta Was this translation helpful? Give feedback.
-
|
I'm sorry if I'm asking something quite basic, but I've not found a clear answer elsewhere. I've followed the directions as closely as possible, but I don't know what to do in this step: "Override the TORCH_COMMAND environment to let SD.Next install torch, torchvision and ipex from local wheels". I've downloaded the files in the correct folder, but I don't know wich file do I have to modify to override the settings. |
Beta Was this translation helpful? Give feedback.
-
|
Just execute the following line in the command line (with which will set the environment variable |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the response, that was the first thing I tried, but I got an error message with the following: "(sdnext) PS C:\WINDOWS\system32\automatic> set TORCH_COMMAND=intel_extension_for_pytorch-2.0.110+git0f2597b-cp310-cp310-win_amd64.whl torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whl
Any idea of what might be wrong? I put the downloaded files in the right folder, and I executed the command within the conda environment, every other step went smoothly. |
Beta Was this translation helpful? Give feedback.
-
|
I finally managed to get it right, I needed to use QUOTATION MARKS -> set "TORCH_COMMAND=intel_extension_for_pytorch-2.0.110+git0f2597b-cp310-cp310-win_amd64.whl torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whl" And it worked just as told. |
Beta Was this translation helpful? Give feedback.
-
|
Can you share tutorial for Windows user? |
Beta Was this translation helpful? Give feedback.
-
|
@TotalDay this thread is the tutorial, just scroll up. |
Beta Was this translation helpful? Give feedback.
-
|
I'm in a weird place. I can't get this to work, though I suspect user error somewhere. I did follow the instructions step-by-step. It installed everything without error. I'm using Linux Mint with an Arc770 16GB by the way. I've slapped that button to make sure the settings are all default, just so we can start with a baseline. Now, if I load an XL model and try to run one 1024x1024 image, the error I get is "RuntimeError: could not create an engine". The traceback call above it gives me this: /media/whackjob/16Tons/STABLE DIFFUSION/automatic/modules/call_queue.py:34 │ |
Beta Was this translation helpful? Give feedback.
-
|
You've installed IPEX for CPU. Use IPEX XPU for GPU support: Removing a docker container doesn't remove the passed folders or volumes. |
Beta Was this translation helpful? Give feedback.
-
|
Also how did you end up with IPEX 1.13? |
Beta Was this translation helpful? Give feedback.
-
|
Brother, I wish I knew. I remember installing 2.0. Well, trying it again. Getting an error. I tried this: Uh, okay. It was 2.0? So I try running that install command again, this one:
And I get the same error as that first block above. So I try to run SD again to confirm I'm screwed before I give up for the night and go do my college schoolwork. And... it runs O_o Still a few errors as it goes: A lot in between of course, but I think that's the important bits. So I pip install 'em. Weirdly, I still get them. And if I try to generate a 1024x1024 image, I get
I'm so close...! I'll keep working on it... if you've got advice, please let me know. I'm also waiting on an ETH delivery for your tip. I doubt we'd ever meet, but I'll buy you a cup of coffee. What's that saying, in Turkiye? One cup of coffee, friend for forty years? |
Beta Was this translation helpful? Give feedback.
-
|
I have now eliminated all errors when SD.Next starts up. I can get in and start generating stuff. Yay! But then it fails at the end of the first generation and then immediately every attempt thereafter. Hrm? Giant wall of code, but it ends with this: I figure, maybe it's a settings thing. I'm googling that error and finding almost nothing. In the UI it's showing this: So why does one image at 1024x1024 immediately eat all my GPU memory? I do not understand. Still working on it... advice welcome of course. |
Beta Was this translation helpful? Give feedback.
-
|
Remove config.json. It's probably using CPU settings right now. Also; |
Beta Was this translation helpful? Give feedback.
-
|
Updated (edited the description, just scroll up): I baked all the dll dependencies ( Users should be able to simply launch SD.Next (dev branch for now) with |
Beta Was this translation helpful? Give feedback.
-
|
Updated wheels in dev branch with these, thank you for the guide and wheels. |
Beta Was this translation helpful? Give feedback.
-
|
@Disty0 Actually I don't think we need to add the guide to wiki any more, as it's simple enough ( |
Beta Was this translation helpful? Give feedback.
-
|
Added to wiki: |
Beta Was this translation helpful? Give feedback.
-
|
Well, I'm back to being down. Now I get this:
|
Beta Was this translation helpful? Give feedback.
-
|
That should have been installed with Intel Level Zero GPU drivers. Install necessary packages in the Wiki page for Intel ARC |
Beta Was this translation helpful? Give feedback.
-
|
And here it is! Trying to run gives me this: There's the culprit. However, I can't seem to find any broken packages. Synaptic package manager filtered for broken packages finds nothing. "sudo apt update --fix-missing" seems to do nothing. I'm really at a loss, now! EDIT: I may have blundered my way into a fix. EDIT3: Figured it might just be a setting. Tweaked that around a bit, now I get this: Frustrating!! I feel like I've been fighting for weeks to get this working and stable. Can't do it here. Can't keep your docker version up, because when it crashes and I need to recreate the container, it tells me the '.' container isn't empty. Even after I delete every folder the container used to be associated with. Even tried lazydocker to see what I was doing wrong. I've done memory tests and pulled some ram that did have bad addresses. The rest is OK. I've reinstalled my OS. I'm even trying a different linux OS. I've spent so much time. |
Beta Was this translation helpful? Give feedback.
-
|
Never mind! Disty0, I figured out why I was getting that error with your docker image. And I've fixed it! Long story short, what I was doing was, I had a set of folders moved out from an old version of Automatic1111 that had my models in it. What I was doing was mounting that folder in place of the set of folders that automatic puts in place for models. So what was happening, was, it would work the first time, and then fail every time after, because that folder now wasn't empty! So instead of mounting it in sd-server/models, now I have it going to sd-server/MODEL. Completely different! Now it all spools up beautifully. Oh, and the hilariously named alphas_cumprod error? Well, it was trying to shove an XL model through a regular SD pipeline, even through I had XL selected. I had to move it off, and then back again, in order for it to "take". So, unless my curse is refreshed, I'm good for a bit! Next up, lol, ss_koyha, for training. Eventually. |
Beta Was this translation helpful? Give feedback.
-
Preparations
Intel GPU Driverproperly (but not 4885 nor 4887, see Drivers from 4885 and newer break IPEX for native windows. intel/intel-extension-for-pytorch#442)git.python 3.10(I only built wheels for python 3.10)Starting from scratch
The latest IPEX wheels (built from source by myself, NOT an official release from INTEL) have been added to
devbranch by @Disty0.Arc users should be able to launch SD.Next directly without needing to manually install any extra dependencies!!
Using python: "C:\Users\vfirs\AppData\Local\Programs\Python\Python310\python.exe" Creating VENV: C:\Users\vfirs\projects\automatic\venv Using VENV: C:\Users\vfirs\projects\automatic\venv 16:18:39-690418 INFO Starting SD.Next 16:18:39-694466 INFO Python 3.10.6 on Windows 16:18:39-793142 INFO Version: app=sd.next updated=2023-10-14 hash=fa86cc0a url=https://github.com/vladmandic/automatic.git/tree/dev 16:18:39-828555 INFO Platform: arch=AMD64 cpu=Intel64 Family 6 Model 183 Stepping 1, GenuineIntel system=Windows release=Windows-10-10.0.22621-SP0 python=3.10.6 16:18:39-830556 INFO Intel OneAPI Toolkit detected 16:18:39-831556 INFO Installing package: https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master% 2Bdll-bundle/torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master% 2Bdll-bundle/torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whl https://github.com/Nuullll/intel-extension-for-pytorch/releases/download/v2.0.110%2Bxpu-master% 2Bdll-bundle/intel_extension_for_pytorch-2.0.110+gitc6ea20b-cp310-cp310-win_amd64.whl ...Performance
IPEX native windows is ~20% slower than WSL or Linux for low resolutions. See details by filtering "ipex" at https://vladmandic.github.io/sd-extension-system-info/pages/benchmark.html.
Why did I build IPEX from source (TL;DR)
Issue of current official IPEX wheel
Intel released IPEX wheels for native windows in Aug'23. However, the official IPEX wheel was built without AOT (Ahead Of Time) compilation support, which dramatically degrades the user experience for IPEX windows -- there's a JIT (Just-In-Time) compilation overhead in the order of 10 minutes (depending on your CPU) each time you start the Web UI, load model weights or change image generation parameter. See intel/intel-extension-for-pytorch#399 for details. In my opinion, this makes IPEX not usable on native Windows.
Building IPEX wheel from source with AOT support
To overcome the above issue, community folks (including me) started to build the IPEX wheel from source since Intel didn't provide a clear ETA for an official AOT wheel release.
The IPEX AOT wheel reduces the warmup overhead from ~10 minutes to ~10 seconds as expected! So I'd like to share the wheels that I built with Python 3.10 and Nuullll/intel-extension-for-pytorch@c6ea20b (which took ~4 hours for i9-13900 + 32GB RAM).
oneAPI dependencies
Previously it was common sense that one needed to install the oneAPI components (DPCPP compiler and MKL) system-wide to be able to use IPEX. Recently I found out that we can actually bake all dll dependencies into the python wheel, so that we don't have to manually install oneAPI any more. You can check repack_wheels.bat to see how I re-package the torch and ipex wheels.
Below is outdated
Install Miniconda (or Anaconda)
Miniconda3 - Python 3.10installer for Windows: https://docs.conda.io/en/latest/miniconda.html#windows-installersIt's highly recommended to use a conda virtual environment instead of SD.Next's default virtualenv. We will need to install some dll dependencies (e.g., oneAPI components, libuv) and conda will automatically put the corresponding dlls into the library path.
Prepare conda virtual environment for SD.Next
Anaconda Prompt (miniconda3). At this time you will be in thebaseenvironment.sdnext)Install dll dependencies in the conda environment (
sdnext)libuvOtherwise, you may encounter the following error when trying to
import torchlater:These are the minimal requirements for running IPEX. And we don't even have to activate oneAPI environment with
setvars.bat-- all required dlls will be put into the library path when we activate thesdnextconda environment.Prepare SD.Next folder
Then put all the downloaded wheels into the SD.Next root folder
automatic\Launch SD.Next with
launch.pydirectlyMake sure you're in the
sdnextconda environment.Override the
TORCH_COMMANDenvironment to let SD.Next install torch, torchvision and ipex from local wheels, then we're all set!Logs:
Re-launch SD.Next later
Simply launch an
Anaconda Prompt (miniconda3). Then activate thesdnextconda environment and executelaunch.pywith--use-ipex.You may notice the following false alarm:
The reason is that there's no
sycl-lsexecutable (it's not necessary at all) inside the conda environment, but we indeed have all the required oneAPI libraries. So just ignore it. I'll file a PR to remove this false alarm.Beta Was this translation helpful? Give feedback.
All reactions