Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi GPU Problems #92

Open
ZSA010807 opened this issue Jul 15, 2024 · 0 comments
Open

Multi GPU Problems #92

ZSA010807 opened this issue Jul 15, 2024 · 0 comments

Comments

@ZSA010807
Copy link

Excuse me,when I train the pvrcnn_st on nuscenes with a single GPU, everything works fine. However, when I use the following command for multi-GPU training, the code fails to run, and the GPUs are not utilized.
The command :
"bash scripts/dist_train.sh 8 --cfg_file cfgs/da-waymo-nus_models/pvrcnn_st3d/pvrcnn_st3d.yaml --batch_size 16 --pretrained_model /data/zhaoshenao/code/ST3D/ckpt/pre-models/da-waymo-nus/st3d/st3d_pvrcnn.pth"

The code gets stuck at this point:

FutureWarning,
WARNING:torch.distributed.run:


Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.


I can't figure out what the problem is. Can you give me some suggestions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant