tf.keras.backend.sqrt(tf.constant(-1.0)) is 0 which is misleading and tf.sqrt(tf.constant(-1.0)) is 'nan' which is the way it should be.

[gaddekumar]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
• TensorFlow installed from (source or binary): Source
• TensorFlow version (use command below): 1.15
• Python version: 3.7
• Bazel version (if compiling from source): No
• GCC/Compiler version (if compiling from source): No
• CUDA/cuDNN version: No
• GPU model and memory: Run in CPU
  Describe the current behavior
  tf.keras.backend.sqrt(tf.constant(-1.0)) returns 0 as clip_by_value is being done in the source code (Which is highly misleading, as can be seen only in the source and not in the function document) whereas tf.sqrt(tf.constant(-1.0)) returns 'nan' which is the expected behavior of any sqrt function. This causes some bugs which are very difficult to track.

Describe the expected behavior
Make sqrt functions return only the expected behavior and remove the clip_by_value.

Code to reproduce the issue
import tensorflow as tf
tf.enable_eager_execution()

tf.keras.backend.sqrt(tf.constant(-1.0)).numpy()
tf.sqrt(tf.constant(-1.0)).numpy()

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

[gowthamkpr]

As you can see from the source code tf.keras.backend.sqrt clips the input value to zero if its less than 0 and to infinity if its equal to infinity as shown by the function here. This is the reason why the output is zero

But in tf.sqrt the sqrt function is from gen_math_ops.py which has a different behaviour i.e., behaviour similar to numpy and thats the reason why its value is defaulted to NaN

[gaddekumar]

But developers usually expect the tensorflow to return nan when tf.sqrt( negative numbers) is done, as thats the norm unlike returning 0, which defies the name of the function. Or, I would request you to update the documentation just to avoid the confusion as tensorflow is moving towards keras.

[gowthamkpr]

Sure @saikumarGadde

[abhiverma1924]

@MarkDaoust Is this issue is solved? Thanks.

[MarkDaoust]

@saikumarGadde: Yes, you have a point but: we need to be very conservative about changing the behavior of existing functions. Also and what is the goal of using tf.keras.backend at all?

At this point tf.keras.backend should be considered an implementation detail. It was a tools for writing keras to be backend-agnostic. TensorFlow is effectively the only backend now.

That's why fchollet removed the documentation for these in:

91e5ad0#diff-e329ed6b8d30dca9a441689005047035

https://www.tensorflow.org/api_docs/python/tf/keras/backend?version=nightly

The functions will still be available in the package but they are undocumented to communicate that they should not be used.