How do I get the training scrips to utilize all my GPUs?

[ohmeow]

Hello there,

I'm running this script:

ACCELERATE_LOG_LEVEL=info accelerate launch --config_file recipes/accelerate_configs/multi_gpu.yaml --num_processes=1 scripts/run_sft.py recipes/zephyr-7b-beta/sft/config_lora.yaml

... but on my machine with 2x3090s ... only GPU 0 is being utilized.

What do I need to change to utlize both of my 3090s for the training run?

Thanks

[ohmeow]

... and of course I just found my answer (set `--num_processes=<# of GPUs to train on>'). :)