Why multi-GPU training not supported for object detection?

[wolfshow]

Is there any reason why multi-GPU training not supported?

[SdustZhangzhen]

Some questions ask for you. How do you use multiple GPU operations? Which parameters need to be modified in train.py?

[roadcode]

you can set the parameter num_clones=YOUR_GPU_NUMS

[SdustZhangzhen]

@roadcode Thinks for your help. It's effectual.

[wolfshow]

you can set the parameter num_clones=YOUR_GPU_NUMS

Thanks @roadcode ! Is it possible to use multiple gpus with the new model_main.py?

[roadcode]

@wolfshow probably not, you can see this issue #5421

[pkulzc]

Please see my answer here.

[jillelajitta]

Sorry to hijack this post
Hi @pkulzc ,
I'm using model_main.py for training. This file is evaluating for every 1200 steps but I want to evaluate for every 50k steps. I tried changing config file but no success. Could you please help me how to evaluate the model for every 50k steps?
My train set consists 700k images
val set consists 300k images
no.of.classes=100
Evaluation time is taking too much.
Thanks.

[mmxuan18]

@pkulzc why not fix this problem, the new main_train.py not support multi-gpu when slim is not use anymore

[pkulzc]

@mlinxiang I didn't understand your question. For now you can only use multi-gpu with train.py. When we migrate to keras in the future, model_main.py will support multi-gpu as well.

[Tantael]

Any progress on this task? Object detection training is bit useless on single GPU. Legacy train seems not working anymore.

[pkulzc]

This is still ongoing. Moving to keras is a huge effort so it'll take some more time.