How Much VRAM Required to Fine Tune XTTS V2 #3268
Replies: 5 comments 6 replies
|
|
|
Did you try float16? |
|
I think it is default to fp16. should i go for bf16?
|
|
@Priyanshu-hawk I recommend to update nvidia driver. There new feature "CUDA – Sysmem Fallback Policy” in nvidia settings. I now on 3070 8Gb easilly finetuning. VRAM using show up to 17Gb in Task Manager. It's use half of RAM for VRAM. Really helped me a lot. On old nvidia driver I also previously got error pytorch like you |
|
Hello @Tessory, i'm using linux mint as my OS.
I'm using I saw on https://developer.nvidia.com/rdp/cudnn-archive that Cudnn - 8.9.2 releases on June 1st 2023 And one more cudnn is available which is 8.9.5 and releases on October 27th 2023. So should i go from Cudnn 8.9.2 to 8.9.6? |
|
Sorry, I don't know about linux drivers. I use it on windows. For windows that new feature just appear in latest november drivers. You can find CUDA – Sysmem Fallback Policy under ‘Manage 3D settings’ in the Nvidia driver settings. It's really did magic for me. XTTS fine-turn training now possible on my 8Gb VRAM card. Just google "CUDA – Sysmem Fallback Policy". Now a lot of people talking about it on many forums. |
|
Ok I'll try to do it for linux |
|
Hey, I'm trying to fine-tune the XTTS_v2 model, and even setting batch_size = 1 and other configurations, I'm getting an out-of-memory error due to lack of VRAM. I have 8GB of VRAM, and I trained the model for Tortoise without any problems. It seems to me that the code might have been incorrectly rewritten... |
|
If you training on Windows, then try to update nVidia driver to latest. There new feature CUDA – Sysmem Fallback Policy under ‘Manage 3D settings’ in the Nvidia driver settings. With that feature GPU now can use CPU RAM. I also got that pytorch error like you, but with new drivers all fine-turned training fine. My GPU also 8Gb VRAM. Task manager show that GPU used up to 17Gb (VRAM+RAM). Really impressive. I'm ready to shake Jensen Huang hand for such a cool feature! |
|
Indeed, I updated the driver and now the training is working; previously, it was doing 3 steps, and now it completed 1 epoch. Thanks a lot! |
where can I change? |
Uh oh!
There was an error while loading. Please reload this page.
I am using RTX 3060 12GB VRAM.
I am trying fine tune a xtts v2 with its default settings.
xtts_v2 training file - https://github.com/coqui-ai/TTS/blob/dev/recipes/ljspeech/xtts_v2/train_gpt_xtts.py
I got Out Of Memory Error with:
I got Out Of Memory Error with:
It is Working with
And I think BTACH_SIZE=1 is very bad for fine tuning.
So, Pls guide me what settings should i change to fine tune with batch_size 2 or 3 on. Or is it possible to fine tune with batch_size 2 or 3 on GPU With 12Gbs of VRAM?
All reactions