[BERT/TensorFlow1.15] What is the speed of multi-GPU on one machine without horovod?

[guotong1988]

Related to Model/Framework(s) or something else (describe)

TensorFlow 1.15

Is your feature request related to a problem? Please describe.
No.
I just want to know the speed result for BERT pretraining of multi-GPU on one machine without horovod

Additional context
It is a feature request and a question:)

[swethmandava]

Do you mean the speedup comparison of horovod vs tensorflow distributed for multi gpu? We haven't tried tensorflow's distributed strategy for Bert Tensorflow but based on some internal experiments on TensorFlow2, we observed horovod's performance is better and hence don't have it on the roadmap.

Any reason you want to run without horovod?

[guotong1988]

horovod is a little hard to install.

[swethmandava]

We recommend you use our containers

[guotong1988]

Could you please give me a docker image with:

tensorFlow 1.15
cuda10
horovod
python3

Thank you very much!

[swethmandava]

Linking release notes for your reference. Thanks and closing the issue