Is torch 2.0 any better than torch 1.13.1 #6455
Replies: 23 comments 128 replies
-
|
torch 1.13.1 is not as good. I have been using 1.31.1 since this repo existed, and I only see the performance improvement when I upgraded to 2.0 AND compiled xformer with 2.0. Did you compile xformer with 2.0 too? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I compiled xformer with 2.0 when I used 2.0. Assuming you meant 1.13.1 and not 1.31.1 I don't see how you could have been using this since the repo existed. |
Beta Was this translation helpful? Give feedback.
-
|
I started using this repo in late Oct. It was 1.13.0 back then. Then I upgraded to 1.13.1 in mid nov. The performance gain I have from 1.13.1 to 2.0 is very reproducible. See #5962 |
Beta Was this translation helpful? Give feedback.
-
|
Well I can get .6 seconds per image without Torch 2.0. Generated 64 images in 38.854778 seconds |
Beta Was this translation helpful? Give feedback.
-
|
I wonder if @brkirch can weigh in here and say if upgrading to Torch 2.0 could make a meaningful difference on macOS + Apple Silicon, too. |
Beta Was this translation helpful? Give feedback.
-
|
I do have an experimental configuration ready. To try it, make sure Xcode is up to date, run Expand#!/bin/bash
#########################################################
# Uncomment and change the variables below to your need:#
#########################################################
# Install directory without trailing slash
#install_dir="/home/$(whoami)"
# Name of the subdirectory
#clone_dir="stable-diffusion-webui"
# Commandline arguments for webui.py, for example: export COMMANDLINE_ARGS="--medvram --opt-split-attention"
export COMMANDLINE_ARGS="$COMMANDLINE_ARGS --opt-sub-quad-attention"
# python3 executable
#python_cmd="python3"
# git executable
#export GIT="git"
# python3 venv without trailing slash (defaults to ${install_dir}/${clone_dir}/venv)
venv_dir="venv-torch-2.0-alpha"
# script to launch to start the app
#export LAUNCH_SCRIPT="launch.py"
# install command for torch
export USE_DISTRIBUTED=1
export TORCH_COMMAND="pip install git+https://github.com/brkirch/pytorch@ebfccd2b78f85bae16e8fcaea1e492abbf395baf#egg=torch --pre torchvision==0.15.0.dev20230106 -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html"
# Requirements file to use for stable-diffusion-webui
#export REQS_FILE="requirements_versions.txt"
# Fixed git repos
#export K_DIFFUSION_PACKAGE=""
#export GFPGAN_PACKAGE=""
# Fixed git commits
#export STABLE_DIFFUSION_COMMIT_HASH=""
#export TAMING_TRANSFORMERS_COMMIT_HASH=""
#export CODEFORMER_COMMIT_HASH=""
#export BLIP_COMMIT_HASH=""
# Uncomment to enable accelerated launch
#export ACCELERATE="True"
###########################################
The first time you run Note that this uses a custom fork: brkirch/pytorch which uses kulinseth/pytorch with a few commits I added on from the main branch in order to get training working. Performance at the time of writing this should usually be more than 10% faster than nightly builds, and usually more than 25% faster than PyTorch 1.12.1 that is currently being used. This should in theory work for everything that already works on PyTorch 1.12.1, including training embeddings and hypernetworks. I'll probably advertise this a bit more if I get #6510 completed and |
Beta Was this translation helpful? Give feedback.
-
|
This is great. I'll try over the weekend. 25% performance improvement is huge. Maybe we should open a dedicated thread about this. There would be too much confusion in this one, where everybody is talking about Windows systems. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
I was able to update the wubui to torch==1.13.1+cu117 and torchvision==0.14.1+cu117 (Windows 10 + Python 3.10.9) Now, Xformers doesnt work anymore after this update.. I tried to rebuild Xformers, to look for new wheels, etc etc etc... I have literally 7 hours trying to update Xformers to 0.0.16 and Im not able to do it, I have searched all github, reedit, google.. followed 21374628746 tutorials, steps... and nothing... I just cant... and I need Xformers because i just have an 1660ti 6gb vram, so Xformers helps me a lot... Anyone knows how to do it? and please, something for dummies, im not an expert as you can see... thanks in advance... my head is going to blow... |
Beta Was this translation helpful? Give feedback.
-
|
Got it working now, and I'm at the triton error that doesn't matter. The other issue I'm having is that when copying over the dlls, I do not have torch/libsl/ have to create /venv/scripts/torch/libsl. also my scripts folder is /Scripts/ is this an issue? |
Beta Was this translation helpful? Give feedback.
-
|
@playlogitech Thank you very much for the info, I was able to install Xformers 0.0.16 on torch==1.13.1+cu117 and torchvision==0.14.1+cu117. (install -U -I --no-deps xformers==0.0.16rc402). Im going to save this venv folder and now build a new venv with your steps for torch==2.00 and also save that venv. Then im going to start testing if i get improvements on my 1660ti 6gb vram. I will let u all know whats better for this graphic card. PS: also thanks to @aliencaocao and @petalas... ur steps also helped me a lot to uderstand how to do it and how this process works. Im not very good at this :p |
Beta Was this translation helpful? Give feedback.
-
for those getting a SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in positio error, put an r before the path. xformers_windows_package = os.environ.get('XFORMERS_WINDOWS_PACKAGE', r"C:\dist\xformers-0.0.15+f351fee.d20230106-cp310-cp310-win_amd64.whl") |
Beta Was this translation helpful? Give feedback.
-
Hi |
Beta Was this translation helpful? Give feedback.
-
Libsl was a typo... Paste the DLLs on this folder -> (your SD root Folder)\venv\Lib\site-packages\torch\lib |
Beta Was this translation helpful? Give feedback.
-
|
is anyone able to still train after upgrading to torch 2.0? am considering upgrading, but don't want to waste time if it results in the inability to train. |
Beta Was this translation helpful? Give feedback.
-
|
I believe you need to make a change to the code to leverage the torch 2.0 features that greatly speeds training. I don't yet know how to train in A1111. I tried "sd_model = torch.compile(sd_model)" in a1111 but it did not speed up inference. |
Beta Was this translation helpful? Give feedback.
-
|
torch.compile indeed does not speed up inference, but it SHOULD speed up training - but there is an existing bug that prevents training to be ran on torch 2.0 builds (#6465 fixes it). So i guess you can only test after this fix is merged. |
Beta Was this translation helpful? Give feedback.
-
|
Ok gents I have down time and motivation going to give this a go. |
Beta Was this translation helpful? Give feedback.
-
|
Ok, im here with my test and results: OS Name: Microsoft Windows 10 Pro Processor: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz, 2592 Mhz, 6 Core(s), 12 Logical Processor(s) I have 4 Stable Diffusions WebUI venv installed:
Prompt: portrait photo of a asia old warrior chief <- (Random prompt from the internet) Restore faces: NO
Same speeds My conclusions of the test in my config: ( Vanilla = Torch 1.13.1 ) < Torch 2.0.0
Everything was installed correctly following the steps, no errors in the process, just the triton one:
Anyone has a clue about the noise generation in 2.0.0? Could be the --precision full --no-half Arguments not optimized for torch 2.0.0 and/or new Xformers? https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Troubleshooting#green-or-black-screen as I have a 1660 ti, I always use those arguments and always worked. Still working with previous torch versions. I know my Graphics card is not the ideal for SD, but is what I have. I hope this information helps. |
Beta Was this translation helpful? Give feedback.
-
|
I am having some weird issues building xformers with torch2.0 20230106+cu118 now...should be my own environment's issue but there is not even an error message so I'm kinda stuck here, wont be able to report if I have the same noise issue or not for now. I use python 3.9 so the wheels built by @hithereai wont work for me. |
Beta Was this translation helpful? Give feedback.
-
|
I just tested with torch 20230106+cu118 and xformer latest commit compiled using it, no issue. Looks like it is specific to your hardware/software. I am on RTX 3080Ti |
Beta Was this translation helpful? Give feedback.
-
|
Maybe related to my graphics card model and the arguments... |
Beta Was this translation helpful? Give feedback.
-
|
While a good effort it was hard to read to see the bottom line that your times were significantly faster using torch 2. However, the fact the images produced were as different as can be I wonder what else is going on. And if you decide to using some vae, no-half, kind of option to fix the noise generated then that would be a fair comparison either. I have a 4090 and I see no difference between the two torch versions. |
Beta Was this translation helpful? Give feedback.
-
|
My test was on a 1660ti 6 GB VRAM. Arguments: --precision full --no-half --no-half-vae --medvram --opt-split-attention --xformers --opt-channelslast <- in order to generate images using card. Checkpoint: SD 1.5 I think im getting black images/noise on Torch 2.0.0 due to my card model + the Arguments. I got an speed improvement (just a little) with torch2.0.0 + Xformers build for that version. However, im just getting black images or noise. |
Beta Was this translation helpful? Give feedback.
-
|
anyone tried on make those checkups on linux and install this triton too? |
Beta Was this translation helpful? Give feedback.
-
|
I dnt have Linux installed on this machine :( Also, got just noise and black images with both torch2.0.0cu118 with your xformers wheel and torch2.0.0cu117 with this wheel: #5962 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
probably due to --medvram or anything else, im on 4090, sorry bud |
Beta Was this translation helpful? Give feedback.
-
Could be the --precision full --no-half Arguments not optimized for torch 2.0.0 and/or new Xformers? https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Troubleshooting#green-or-black-screen as I have a 1660 ti, I always use those arguments and always worked. Still working with previous torch versions. |
Beta Was this translation helpful? Give feedback.
-
|
@playlogitech I have a 4090 also on Ubuntu. Using the v2.1 512 model how long does a batch of 64 images (count=4, size=16) takes with 20 steps and euler_a? Have you tried this with both torch 1.13.1 and torch 2? I see no difference. I only use xformers as a command lipne option. |
Beta Was this translation helpful? Give feedback.
-
|
Looks like I was doing it wrong or maybe It just doesn't work with T4 yet. I was following the instruction from this https://pytorch.org/get-started/pytorch-2.0/#requirements and then clone the xformers from its github, and lastly compile them. and after it was done, I created the wheel file to make sure I would not compile it again. |
Beta Was this translation helpful? Give feedback.
-
|
4090 here
|
Beta Was this translation helpful? Give feedback.
-
|
have u got Speeds improvements using torch 2.0.0? |
Beta Was this translation helpful? Give feedback.
-
|
xformers built on torch 2 is a little faster. that's the only improvement i have seen. |
Beta Was this translation helpful? Give feedback.
-
|
@Zuxier I also have a 4090. I do not see an improvement with torch 2.0. I get: Total progress: 100%|████| 80/80 [00:37<00:00, 2.12it/s] What do you get with torch 2.0? |
Beta Was this translation helpful? Give feedback.
-
no increase of speed from torch2 compared to what i had, but xformers built on torch 2 gave me some extra speed. |
Beta Was this translation helpful? Give feedback.
-
|
I assume you also had a xformers compatible with whatever you had before(torch 1.13.1?). I get 37 seconds for 64 images(4x16) with 1.13.1. What does anyone else with a 4090 get with torch2 and xformers? What does this command tell you(gpu.compute_capability for instance): |
Beta Was this translation helpful? Give feedback.
-
|
Dell 3070 (8GB) OC, Win 10 Home, 16GB DDR4 Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3508515359, Size: 512x512, Model hash: a9263745, Model: v1-5-pruned |
Beta Was this translation helpful? Give feedback.
-
I am using latest Studio Driver + latest cuDNN dlls. Edit: |
Beta Was this translation helpful? Give feedback.
-
Game Driver 528.02 + dlls from cudnn-windows-x86_64-8.7.0.84_cuda11-archive.zip
This helped to get 9.5 instead of 8.5. PYTORCH_CUDA_ALLOC_CONF (added to webui-user.bat) - no effect. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Yes channel last should always be enabled on Turing and newer cards to use tensor core |
Beta Was this translation helpful? Give feedback.
-
|
I've enabled it and generated images and seen no benefit at all. My 4090 certainly should have tensor cores. Also, memory layout is one thing but shouldn't you also need to set something like enable_tf32 also? Would it be a good idea to simply have a micro-benchmark py program(maybe 20 lines) which can test whether some convolution operation was faster with that memory layout? |
Beta Was this translation helpful? Give feedback.
-
|
Tensorfloat32 is by default enabled in this repo. |
Beta Was this translation helpful? Give feedback.
-
|
Gave it a test myself with my RTX 3070 Fresh installs for both: 8850fc2
Torch 2.0.0 + Xformers Compiled with Torch 2.0.0 + cuda116 (No Triton) Torch 1.13.1 + Xformers from Automatic No major change, but it didn't absolutely break things. Edit: Changed some of the Compiler options, set it to just |
Beta Was this translation helpful? Give feedback.
-
|
I compiled xformers with flags below: But couldn't find anything about |
Beta Was this translation helpful? Give feedback.
-
|
this means torch.compile options, nothing to do with xformer or torch building. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, Tested that and got nice boost on single image (16.6it/s) but better result with batch of 10 to 20 (17.1 it/s) on 3070 (OC), I'll do more tests with different settings, will post the results shortly. |
Beta Was this translation helpful? Give feedback.
-
|
Check #6615 for detailed test results. |
Beta Was this translation helpful? Give feedback.
-
|
Hey guys, I'm a novice when it comes to all the lingo, (I was fine 10+ years ago, but a lot has changed that I haven't caught up on...) Is installing torch 2.0 better than the torch I have now (whatever version I was told to install....) My card is a GTX 1080, if I want to install the new torch, what steps do I need to do? Install python whatever, then do I need to change any files in my Auto WebUI folder? I read that xformers is installed automatically when I just type --xformers into my bat file, is that not true, or if it is true... after installing torch 2.0, do I need to manually rebuild/install/whatever? I haven't updated because all the information that I've read/watched has told me autoui was built around python 3.10 (or whatever...) Are we updating python, or just the torch. Does this affect any other extensions I've installed recently? Thanks ya'll! |
Beta Was this translation helpful? Give feedback.
-
|
xformers is far more important than torch 2.0.
I see no perf improvement with torch 2.0 but others claim so.
The problem is that no one quotes image generation times and only it/s.
I won't even try to test torch 2.0 anymore until someone says they can
generate, on a 4090, 96 images (nbatches=6 X batchsize=16) in under 57
seconds. I get this with torch 1.13.1. Torch 2.0 is no faster.
opt-channelslast only slows things down for me.
…On Tue, Jan 10, 2023 at 3:54 PM Croestalker ***@***.***> wrote:
Hey guys, I'm a novice when it comes to all the lingo, (I was fine 10+
years ago, but a lot has changed that I haven't caught up on...) Is
installing torch 2.0 better than the torch I have now (whatever version I
was told to install....) My card is a GTX 1080, if I want to install the
new torch, what steps do I need to do? Install python whatever, then do I
need to change any files in my Auto WebUI folder? I read that xformers is
installed automatically when I just type --xformers into my bat file, is
that not true, or if it is true... after installing torch 2.0, do I need to
manually rebuild/install/whatever?
I haven't updated because all the information that I've read/watched has
told me autoui was built around python 3.10 (or whatever...) Are we
updating python, or just the torch. Does this affect any other extensions
I've installed recently?
Thanks ya'll!
—
Reply to this email directly, view it on GitHub
<#6455 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3YFZABVF2MICWBHUCGFRH3WRXZC7ANCNFSM6AAAAAATTUV6EU>
.
You are receiving this because you authored the thread.Message ID:
<AUTOMATIC1111/stable-diffusion-webui/repo-discussions/6455/comments/4650926
@github.com>
|
Beta Was this translation helpful? Give feedback.
-
|
PleezDeez torch.compile is a newish torch thing and xformers is xformers. I'm not sure what you are looking for. I do rebuild xformers after rebuilding or reinstalling torch. torch.compile is for learning but I'm speeding up inference by correctly building my own torch. In any case, I rebuild xformers with: MAX_JOBS=16 CUDA_PATH=/usr/local/cuda-$CUDAVER pip3 install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers |
Beta Was this translation helpful? Give feedback.
-
|
Maybe it's just my particular setup causing me issues bc I can't get xformers and torch 2.0 to play nicely with each other without errors galore popping up so I'll just wait it out until xformers has better support for it baked in. Tho on the flipside I have Triton working well with xformers 0.16.0dev424 in windows thanks to a file shared the other day and I was able to use that to put together a whl of deepspeed from it that also works... I hope... It built it and installs at least... I have no way to test it bc a new error that I traced back to accelerate anytime distributed training at all is turned on in windows even from a fresh install of A1111 on a different drive is blocking my progression and I'm a builder, literally build houses for a living, not a coder so am already well outside my whl house 🤷🏻♂️ |
Beta Was this translation helpful? Give feedback.
-
|
You have to build xformers yourself, if you didnt |
Beta Was this translation helpful? Give feedback.
-
|
@aifartist I have a 4090, too. But it is not working that fast. How did you do that? Only with rebuilding xformers? Could you explain a step by step guide (for dummies) how to rebuild and install xformers? |
Beta Was this translation helpful? Give feedback.
-
|
@hippopotamus1000 Are you on Ubuntu or WIndows? |
Beta Was this translation helpful? Give feedback.
-
|
Status for anybody interested ... Premise 2: If you build Torch 2 locally you can get a 3X perf improvement on a 4090. Something is wrong with the nightly builds that pytorch.org provides. I had claimed this before and today someone else has just confirmed this. I was contacted on Facebook by an ex CTO that had access to various GPU's in the cloud and offered me hardware to test and his expertise in exchange for my help in speeding up his Linux based SD service. Spent 12 hours online figure out how to build Torch 2.0 locally and when it worked it was indeed as fast as I hoped for. We also confirmed some speed up on a 3090 and tomorrow he will test an A4000 GPU which currently is only doing 7 it/s. Because I spent all day doing this I'm still trying to figure out why I can't build Torch 2.0 for a second time so I can document all the steps to give to everyone on LInux. |
Beta Was this translation helpful? Give feedback.
-
I helped someone else creating a SD service in the cloud to build Torch 2.0 on RedHat and it worked great speeding up image generations to 35+ it/s. The difference is he is using DiffUsers. If I log onto his machine with a 4090 which is using our local Torch build where he gets the great performance, install A1111 and then install out local Torch for it I get the same SEGV. Something in A1111 is no longer working with the local build. But diffuser doesn't have that problem. Good luck to you but I need to debug what is happening. I've been getting up to speed on cmake build scripts to see why it is having problems with identifying the GPU. I've learned a few things about CUDA installation that may be helpful but will first get back to figure this out. |
Beta Was this translation helpful? Give feedback.
-
I may have mentioned before but CUDA 11.8 is the first to leverage the sm_89 Ada Lovelace architecture(4090 and others). This will definitely help 4090's. |
Beta Was this translation helpful? Give feedback.
-
cuDNN dlls are a newer version, they just perform better. |
Beta Was this translation helpful? Give feedback.
-
When using your compiled xformers I am getting quite a bit less it/s than when I built them. |
Beta Was this translation helpful? Give feedback.
-
It's not me claiming those speeds, to me its just few % faster than older torch/xformers. Of the wheels i built some should be faster, but dreambooth training wise only the 0.14.dev0 works for me. 10it/s on a single image with a 4090 is slow tho. |
Beta Was this translation helpful? Give feedback.
-
|
Well, this is fun! NOT!!! |
Beta Was this translation helpful? Give feedback.
-
|
it seems you generating on cpu |
Beta Was this translation helpful? Give feedback.
-
|
@playlogitech I had that same thought although that'd be quite fast but I do have a 32 processor i9-13900. However, in the venv for the a1111 I get the following for torch version and cuda is_available(): |
Beta Was this translation helpful? Give feedback.
-
|
So this is a long discussion, but I will post random observations from CachyOS Linux/a 2060 laptop:
|
Beta Was this translation helpful? Give feedback.
-
|
@hippopotamus1000 |
Beta Was this translation helpful? Give feedback.
-
|
WOW! Looking forward to the explanation. |
Beta Was this translation helpful? Give feedback.
-
|
@outpoints @hippopotamus1000 5.17.0-1019-oem #20-Ubuntu SMP But I need to sleep. Tomorrow I hope to be able to wrap this up. |
Beta Was this translation helpful? Give feedback.
-
how much is cpu involved in those speeds? I start thinking my 5900x bottlenecking my it/s, thats why im at 25-28 it/s |
Beta Was this translation helpful? Give feedback.
-
|
i'm catching up fast with torch 2.0 and deep speed. and cu117 edit you ~/.bashrc file and add : save and run : source ~/.bashrc 1 : pip install numpy --pre torch[deepspeed] torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu117 deep speed : has good config examples on how to optimize the training using hugging face transformers ... **Accelerating launch.py... To create a public link, set |
Beta Was this translation helpful? Give feedback.
-
|
Started discussion #6932 |
Beta Was this translation helpful? Give feedback.
-
|
I would have preferred to have this thread under my own name as I would like to have more visibility in the community once I get up to speed on SD. But, no problem. I just woke up and will start putting together a HOW-TO document for Linux users. |
Beta Was this translation helpful? Give feedback.
-
|
@aifartist you could create your own discussion if you'd like as you probably have other important pieces to add that would be harder to find as GitHub discussions is a mess |
Beta Was this translation helpful? Give feedback.
-
|
The performance differences in pytorch has more to do with the libcudnn version than anything else. I have found the root cause of why some see 13 it/s on a 4090 and others see ~40 it/s. See #6954 I've let the pytorch community know. |
Beta Was this translation helpful? Give feedback.
-
|
Torch 2.0.0 is >>NOT<< a lot faster, faster nor even a little bit faster than 1.13.1. If I am wrong, show me results where you get over 45 it/s with a regular 4090. Torch 2 is NOT faster. Now xformers does give me a 28% speedup over the 30.5it/s I see when it is not used. That is a real example of a lot faster. I get very consistent results. Of course, there might be some edge case that isn't what someone doing normal image generation will see. Maybe by GA in Mar? we'll see some improvements with inference. |
Beta Was this translation helpful? Give feedback.
-
|
i agree, torch 2.0 is not any better on its own than torch 1.13.1 if everything is configured correctly torch 2.0 has a long term potential due to inclusion of |
Beta Was this translation helpful? Give feedback.
-
|
Many people out there including myself will disagree with xformer part. Xformer + torch 2.0 has observable performance improvements for us. Your machines may just be too fast to see it. |
Beta Was this translation helpful? Give feedback.
-
quite possibly, it just has been my experience - particularly when ppl use older if you run |
Beta Was this translation helpful? Give feedback.
-
|
I get 10% perf boost on windows with functorch and triton UNAVAILABLE |
Beta Was this translation helpful? Give feedback.
-
|
How would I get this to work for the new launch.py where line 182 and 242 are the relevant lines? The xformers install is different. Clean install of windows and I'm starting from scratch. |
Beta Was this translation helpful? Give feedback.
-
|
You can set TORCH_COMMAND to be blank in the user .bat file. xformers has to be built from source pretty much every time you update torch 2.0, but the ui wont try to reinstall it once you build it. |
Beta Was this translation helpful? Give feedback.
-
|
24.) Automatic1111 Web UI - PC - Free |
Beta Was this translation helpful? Give feedback.
-
|
For my setup I can confirm the superiority of torch 2.0 / cuda 11.8 over torch 1.13.1 / cuda 11.7. This seems to be mostly due to improvements in cuda 11.8 which is supported by torch 2.0 (source). Performance went up from 5.97 it/s to 8.59 it/s while VRAM usage being much lower. I wasn't using xformers so that explains the larger gain. System: |
Beta Was this translation helpful? Give feedback.
and others
-
Yesterday I was experimenting with building torch 2.0 so that I could ALSO use CUDA-12.0. I was excited it was so fast. Today I explored deeper and to do so I did a clean install of A1111 and it was very fast without torch 2. ??? I found an older A1111 I still had and it was slow. After a lot of work to figure out what the difference was I found the old version had torch 1.12.1+cu113 and if you remove and reinstall you get 1.13.1+cu117.
Indeed it was much faster. I now have learned that "git pull" to update A1111 doesn't upgrade python packages to newer versions.
Then I installed torch 2.0.0.dev20230106+cu117 and I no longer see a perf improvement over 1.13.1. I have a feeling that CUDA-12.0 won't make any further difference. A valuable lesson learned.
I can still get another 16% perf improvement with the two other changes I mentioned in another post but now torch 2.0 has nothing to do with it. I was going to post those changes today but trying to figure out why the baseline was so much faster has consumed most of the day.
@hippopotamus1000
@DustyCooper
@aliencaocao See if torch 1.13.1 is just as good as torch 2.
Beta Was this translation helpful? Give feedback.
All reactions