[docs] (non) determinism in TF

[mpekalski]

Describe the feature and the current behavior/state.
A documentation explaining determinism and sources of non determinism in TF applications. It should cover things from having multiple readers, seeds in the specific ops, seed in numpy, PYTHONHASHSEED, non-deterministic ops in CUDA (and why TF does not support passing deterministic flag even though it would be slower), TF_CUDNN_USE_AUTOTUNE, and probably some others I do not know about.

Who will benefit with this feature?
I take part in many Kaggle competitions and very often people there have problems with non-determinism in NN.

[jvishnuvardhan]

@mpekalski Can you please check this https://www.tensorflow.org/api_docs/python/tf/config/experimental/enable_op_determinism. This will enable your code to run deterministically at the cost of some computational time. Thanks!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.