RFC: Enabling TensorFloat-32 by default #287
Conversation
reedwm
requested review from
ematejska,
ewilderj,
martinwicke and
theadactyl
as code owners
|
|
||
| ## Design Proposal | ||
|
|
||
| The design is to flip the default value of [`tf.config.experimental.enable_tensor_float_32_execution`](https://www.tensorflow.org/api_docs/python/tf/config/experimental/enable_tensor_float_32_execution) from False to True. |
There was a problem hiding this comment.
Would it be practical to provide a user control for setting the default type with a global toggle, for all numerical ops? That might be useful in case there isn't a one-size-fits-all.
There was a problem hiding this comment.
This is somewhat orthogonal to TensorFloat-32, since TF32 is not a TensorFlow dtype. Even with TF32 enabled, tf.linalg.matmul outputs float32 tensors when passed float32 inputs. The only effect of TF32 is increased performance and inputs to ops like matmul are rounded from 23 bits of precision to 10 bits of precision.
There should be an API to set the default dtype for ops like tf.ones that don't take an input tensor. But I think ops like matmul should continue to unconditionally output tensors of the same dtype as the inputs.
This section summarizes what was discussed during the design review.
|
Design Reive Notes were added to the RFC document. This is accepted. |
ematejska
added
RFC: Accepted
4 participants
This RFC will be open for comment until October 14, 2020. The comment period is longer than usual in order to give a lot of time to investigate and test TF32.
Enabling TensorFloat-32 by default
Objective
Enable TensorFloat-32 by default.
This will improve performance but lower numeric precision for certain ops like matmul and convolutions by default.