Amd videocard optimization

[ppleaser]

I use this commands:
set COMMANDLINE_ARGS=--medvram --opt-split-attention

How can I optimize the generation even more for the 6600 xt graphics card. Right now, 512x768 images take up 7.8/8 gb of memory, generation speed is about 1.2 it/s. Are there any other ways to increase the generation speed, or reduce the amount of video memory consumed without major negative consequences? Maybe there is a more suitable repository optimized for amd video cards?

[49RpK5dY]

Try --opt-sub-quad-attention. It won't increase the speed but it sure saves a lot of ram. On rx5700 with 8Gb it can generate in 1024x576 with no problem.

[gigi798]

switch to linux, it'll give you better it/s

[ApoorvRChincholkar]

[gigi798]

try this to fix memory issues #6722

[ApoorvRChincholkar]

Hey @yamfun try this, it worked very well for me and completely eliminated the stutters and other issues. Also eliminated drag-and-drop issue that you might have noticed on Ubunty 22.04

https://itsfoss.com/switch-xorg-wayland/

Wayland is buggy, use XOrg.

[yamfun]

seems like the google perftools made a difference, I can generate way more before it hangs (like from every hour to once per night). Thanks!

[ApoorvRChincholkar]

Oh right, I had installed that wayy back. Even had forgot about those.

[Fuchsei]

set COMMANDLINE_ARGS= --precision full --no-half --always-batch-cond-uncond --opt-sub-quad-attention --disable-nan-check
This was mine for a while, I have worse results tho but maybe some of them could help you

[elen07zz]

i can leave precision full and nohalf, im using rx 6700

source venv/bin/activate
export HSA_OVERRIDE_GFX_VERSION=10.3.0
export LD_PRELOAD=libtcmalloc.so.4
export PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128
export SAFETENSORS_FAST_GPU=1
python3 launch.py --opt-split-attention --upcast-sampling --disable-nan-check --opt-sdp-attention --medvram --always-batch-cond-uncond

[set-soft]

Hi @elen07zz !

export PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128
export SAFETENSORS_FAST_GPU=1

Do you really measure some change when using these options for an AMD board? I'm asking because I didn't see any change on my RX5500XT and when I searched about them I only found references to CUDA.

[Sakura-Luna]

The first line is valid, the second line is no longer needed.

[set-soft]

The first line is valid, the second line is no longer needed.

Are you sure? The PyTorch docs doesn't mention it for the HIP semantics, only for CUDA semantics. And even the out-of memory messages says PYTORCH_HIP_ALLOC_CONF, which seems to be something not implemented.

[Sakura-Luna]

PyTorch mixes a lot of cuda interfaces without documentation annotations, so the documentation may not be reliable. In theory, PyTorch should have implemented garbage collection, but I am not sure.

[howDoILearnPython]

i would like to share my experience here
Rather than AMD video card, i am using 3200G with vega 8 on board display. So it's virtually equal to "NO GPU" Believe it or not? my maximum achievable resolution is 512X768 with 9.5s/it, or 512x512 with 5s/it
My argument used is
--medvram --listen --share --no-gradio-queue --opt-sub-quad-attention --sub-quad-q-chunk-size 256 --sub-quad-kv-chunk-size 256 --sub-quad-chunk-threshold 70

sorry for the chinese windows interface, surprisingly, you can see my stable diffusion not only use the dedicated VRAM, but also the shared memory. So total of 19.2/20GB VRAM used. this only happen when i allocated 8GB as dedicated VRAM BIOS. It's not gonna happen if 4GB or 16GB is allocated.

[ImBehinYa]

Did you set somthing so it uses your RAM to?

[ApoorvRChincholkar]

Many, many thanks to you! With your commandline args I can now easily generate 800x1200px images (maybe even higher, need to test). Earlier I was strictly limited to 640x960px with medvram. With lowvram I could go above, but that was excruciatingly slow. Your args however keep this very fast and with no loss in quality! Saved me much of my time.

Using 6600xt.

[AlexeyJersey]

HUGE Thanks to you!
You made my day. No annoying memory issue.

Tested on:
darkphonix3d 1024x1024 txt2img
juggernautXL 512x512 txt2img

my gpu is TX5700XT 8gb

[yamfun]

OT,
when you guys use WD captioning in A1111 or in Kohya, do you guys get TensorFlow error becoz we are actually usng AMD TensorFlow-rocm? Currently I make another separate folder set of A1111/Kohya and install the default TensorFlow there and just start that instance for captioning. Do you guys have proper workaround for that?