TESLA V100 Training LORA not use 100%GPU

[littleyeson]

In my LORA training process, my GPU utilization always remains around 30-40%, as well as the TDP. The training speed is extremely slow. How can I resolve this? Additionally, my machine has two other GPUs, V100 and 2080ti 22G, but it always defaults to using V100 and I am unable to select the other one

[littleyeson]

I solved the problem of switching GPUs. In the setup.bat settings,choose 4 accelerate config,at last instead of entering 'all' for selecting GPUs, input the number of GPU IDs. For example, if there are two GPUs, numbered 0 and 1, input '0,1'.

[littleyeson]

I find use adamW8bit and more bitchsize (GPU memory not overflow).The GPU load can use 70~80%.