tf.distribute.experimental.CommunicationOptions does not work at all

[Flamefire]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): 2.4-rc1
• Python version: 3.7

Describe the current behavior

The implementation of tf.distribute.experimental.CommunicationOptions tries to use some trickery (for unknown reason) and fails to do anything in its constructor. See

tensorflow/tensorflow/python/distribute/collective_util.py

Line 84 in bd3a525

return Options.__new__(Options, *args, **kwargs)

and

tensorflow/tensorflow/python/distribute/collective_util.py

Line 114 in bd3a525

pass

This makes it impossible to use the non-deprecated tf.distribute.MultiWorkerMirroredStrategy and specify the communication backend as that requires to create such an options instance

Describe the expected behavior

Creating an instance of that class initializes the members

Standalone code to reproduce the issue

import tensorflow as tf
tf.distribute.experimental.CommunicationOptions(bytes_per_pack=-1) # This should fail!
communication = tf.distribute.experimental.CommunicationImplementation.NCCL
o = tf.distribute.experimental.CommunicationOptions(implementation=communication)
assert o.implementation == communication # Error: no attribute 'implementation'

Other info / logs

IMO this is critical for the 2.4 release and the fix is rather simple: Don't have an extra Options class but simply put that code into _OptionsExported and let Hints derive from that. To not disturb other (internal) code renaming _OptionsExported to Options would be wise.

[crccw]

It's to separate internal APIs and external APIs (e.g. the merge() method). A fix is in the pipeline and will be cherrypicked.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No