Sarkars/logsoftmax

[sayantan-nervana]

No description provided.

[sindhu-nervana]

LGTM

[sindhu-nervana]

TF python tests seem to be failing with tolerance errors

AssertionError:
--
  | Not equal to tolerance rtol=1e-05, atol=1e-05
  | Mismatched value: a is different from b.
  | not close where = (array([], dtype=int64), array([], dtype=int64))
  | not close lhs = []
  | not close rhs = []
  | not close dif = []
  | not close tol = []
  | dtype = float64, shape = (2, 4)
  | x and y -inf location mismatch:
  | x: array([[-1.386294e+000, -1.386294e+000, -1.386294e+000, -1.386294e+000],
  | [ 0.000000e+000, -1.797693e+308, -1.797693e+308, -1.797693e+308]])
  | y: array([[-1.386294, -1.386294, -1.386294, -1.386294],
  | [     -inf,      -inf,      -inf,      -inf]])

[sayantan-nervana]

Investigated one of the test failures softmax_op_test.SoftmaxTest.testAlongFirstDimension:

For this input:
max = np.finfo(type).max # pylint: disable=redefined-builtin
features = np.array([[1., 1., 1., 1.], [max, 1., 2., 3.]]).astype(type)

Tf produces: np.array([[-1.386294, -1.386294, -1.386294, -1.386294],
[0, -max, -max, -max]])
We produce: y: array([[-1.386294, -1.386294, -1.386294, -1.386294],
[ -inf, -inf, -inf, -inf]])
Numpy produces:
array([[-1.38629436, -1.38629436, -1.38629436, -1.38629436],
[ -inf, -inf, -inf, -inf]])
if we do p - log(sum(exp(p)))

But the actual answer should indeed be [0, -max, -max, -max].

[shresthamalik]

LGTM