whats the minimum requirments for finetuning? (not lora) a 5090?

[XDM-Inc]

 [rank0]: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 9.53 GiB. GPU 0 has a total capacity of 23.52 GiB
[rank0]: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 9.53 GiB. GPU 0 has a total capacity of 23.52 GiB of which 6.39 GiB is free. Process 16265 has 103.09 MiB memory in use. Process 16790 has 102.75 MiB memory in use. Process 1626306 has 245.23 MiB memory in use. Process 1870714 has 386.00 MiB memory in use. Including non-PyTorch memory, this process has 15.70 GiB memory in use.

i just swapped out my radeon 7900xtx for a 4090 because i was told that rocm requires more vram for the same training and now it seems like it does not matter. my 24gb of vram is STILL not enough. i have been trying a ton of settings to lower vram and STILL OOM. what does it want from me??

[bghira]

what model are you trying to train? that matters a lot

[XDM-Inc]

im trying to finetune from Cyberrealistic XL 9 as i have been doing a while in onetrainer

[bghira]

i've not really bothered with training full-rank in quite some time, it's never given me better results than LoRA (especially rank 128) or LyCORIS LoKr but those older Unet models are harder to optimise for. it doesn't work great with most offload techniques, and you can't use quantisation to hit int8 weights while training the base model. can i ask why you're wanting full-rank training?

if you insist on full rank training, you'll want to try things like:

• attention_mechanism=native-efficient might work better for you
• gradient checkpointing - ensure it's on
• gradient checkpointing interval - ensure it's off
  • only tweak things like this once it actually works for you without OOM
• RamTorch might work for SDXL, but admittedly, i'm not sure i've tested that combination. it's weight streaming from RAM. it's going to slow things down a bunch
  • musubi block swap won't work for SDXL as it's a Unet based model, harder to implement offload for
• activation offload will help, but slows down training a lot

you probably could ask some LLM agent about your OneTrainer settings for SDXL and how to achieve the same in SimpleTuner, but it's just kinda ... better to move onto DiT based architectures like Anima, Krea2, Ideogram4, Z-Image, or even Flux2 Klein.

still, i think you'd be well served to try LyCORIS LoKr - you can make them quite large with a factor of 1 or 2! that's like training the full SDXL weights, but without as much of the heartache. even if you switch architectures, you'll really want to train an adapter instead of full-rank.

[XDM-Inc]

it's never given me better results than LoRA (especially rank 128)

thats amazing. NEVER have i EVER got a lora to look half as good as a full finetune. not once in 3 years and 13 models. loras usually looks "passable" when it comes to Realistic human subjects that is. i forgot to mention that

as for your recommendations i have already tried all of those. i am now as far as trying deep speed with offload but now its memory leaking and exhausting all 64gb of my system ram and getting terminated by linux kernel to prevent system lockup

[bghira]

thanks, i've reported you for impersonating RunPod and spamming.