the number of gpu used when training models

[Counter-F]

Hello, we check these three pretrained models and find they are trained with 2 gpus (shown three figures below).

But we notice that in the https://github.com/open-mmlab/mmsegmentation/blob/master/docs/en/model_zoo.md, you say that you use ditributed training with 4 GPUs by defaults ( as shown below).

We would like to ask what is the reason to use two gpus in training three aforementioned models?

[MengzhangLI]

samples_per_gpu means the batch numbers on each gpu, so if 4 gpus used and samples_per_gpu=2, total batch sizes are 4 x 2 = 8.

Plus, 4 GPUs by default is no longer accurate, for instance, many transformer models use 8 GPUs by default. You can checkout log in each markdown config and checkout detailed infomation.

What's more, in MMSegmentation 1.x, we rename our config names, you could tell information by its name.

https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/swin/swin-tiny-patch4-window7-in1k-pre_upernet_8xb2-160k_ade20k-512x512.py

8xb2 means 8 GPUs x 2 samplers = 16 batch sizes.

[Counter-F]

Sorry to bother again.
What we just wanna know is why gpu_ids= range(0,1), a.k.a only gpu0 is used instead of 4 gpus, typically for these three model. We want to know is there some special reason to set only 1 gpu for those three models?

[MengzhangLI]

You should make total batch sizes the same as 4 gpus, i.e., make samples_per_gpu is 4 times of our original setting.

[MeowZheng]

Sorry to bother again. What we just wanna know is why gpu_ids= range(0,1), a.k.a only gpu0 is used instead of 4 gpus, typically for these three model. We want to know is there some special reason to set only 1 gpu for those three models?

thanks for your bug report. These three models are very old for mmseg, and the config recorded inaccurate gpu_ids when training them

mmsegmentation/tools/train.py

Line 84 in b2724da

cfg.gpu_ids = range(1) if args.gpus is None else range(args.gpus)

but we have fix it in train.py when training model distributedly

mmsegmentation/tools/train.py

Lines 155 to 158 in b42c487

	init_dist(args.launcher, **cfg.dist_params)
	# gpu_ids is used to calculate iter when resuming checkpoint
	_, world_size = get_dist_info()
	cfg.gpu_ids = range(world_size)

I have closed this issue as the bug have been fixed, but if you still have any problems, you can re-open it anytime.