Is training possible on RTX3070 8GB?

[xhost007]

Hi,

I have been unlucky so far. Got the newest pull, tried a lot of different settings and got CUDA OOM all the times. Before I spend a night training on CPU only, has anyone been successful on a 8GB GPU and if so how?

[Slug-Cat]

I think the minimum vram possible would be running with xformers, 8adam, gradient checkpointing, and (No) latent caching enabled. Though, the lowest I've ever heard is about 9gb.

[xhost007]

Well shit. I'll be training CPU at night. 5 hours instead of 15 minutes. Oh well.

[Slug-Cat]

Thanks, Nvidia

[differentprogramming]

I wonder what I did wrong that with those settings it takes almost all of 12 gb vram.
Note, I had to compile xformers myself to get it to install.

[sgsdxzy]

If you have Linux you can try deepspeed to get below 8GB. On Windows I see the lowest reported in around 9.x GB.
You can try cpu only though.

[Nyaster]

Yes is posible if your using linux and deepspeed, in my laptop i have around 5it\s

[Thomas-MMJ]

With LORA have done it on 6GB, had to disable the full checkpoint saves (via editing the code), disable saves of images (also via a code edit), enable 8bit, enable LORA, used an efficient attention (not xformers, the other option), and disable text encoder training.