How to decrease VRAM usage during runtime?

[walkingwithGod2017]

Great repo! I've seen your official demo, and the results are impressive. Thank you for your research efforts.
Now, I'm facing an issue: my GPU has 8GB of memory, I'm using Windows, and during execution, it's using around 15GB of VRAM, causing very slow speeds. It takes about 10 minutes to generate a piece of music. Could you please suggest ways to speed up the generation process?
Looking forward to your response! ^^ ^^

[xushengyuan]

You can wait for future community contributions regarding model quantization. Currently, inference can be run on Linux based on torch.compile, which means it may be possible to attempt using PyTorch's official torchao quantization library for inference.

[walkingwithGod2017]

You can wait for future community contributions regarding model quantization. Currently, inference can be run on Linux based on torch.compile, which means it may be possible to attempt using PyTorch's official torchao quantization library for inference.

OK, thank you for your kind reply. ^^ ^^

[iChristGit]

I am not sure its only 15GB of VRAM,. for me its uses all of my 24GB of vram and then some RAM as well.
You probably spilled like 30GB to ram (?)

[ChuxiJ]

pull the newest code

[walkingwithGod2017]

pull the newest code

OK, thank you for your great job. ^^ ^^