Where are the trainable variables placed in the "Multi-GPU processing with data parallelism " ?

[zhangjcqq]

I mean if there are 4 gpus that can be used for data parallelism. Where are the variables placed ? All variables are placed on the gpu:0 or in some kind of other allocation approach？
If all the variables are placed on the gpu0, it seems possible to meet the OOM (Out of Memory
) issue.
Waiting for your reply, thanks!

[vahidk]

Variables are placed on one GPU.