Support all fp types in GPU SparseTensorDenseMatMul

[benbarsdell]

Adds Eigen::half for CPU, and Eigen::half, double, and complex for GPU.

cc @nluehr

[benbarsdell]

I'm investigating the test failures.

[benbarsdell]

I've fixed the complex type test failures (I just missed #47355) and added support for float32 accumulation for float16 input/output, along with tests for float16.

[allenlavoie]

Ah, the unit tests were soft-placing onto the CPU previously? Looks good.

[benbarsdell]

The tests were running on the GPU, it was just that #47355 changed (fixed) the handling of complex conjugates (on CPU), and my local branch didn't have that commit.

[allenlavoie]

When we build internally I see a bunch of references to quiet_NaN not being supported for complex and complex:

third_party/tensorflow/core/kernels/sparse_tensor_dense_matmul_op_gpu.cu.cc:104:9: note: in instantiation of function template specialization 'tensorflow::SparseTensorDenseMatMulKernel<std::complex<float>, std::complex<float>, int, false, false>' requested here
        SparseTensorDenseMatMulKernel<T, Tsum, Tindices, ADJ_A, ADJ_B>,
        ^
In file included from third_party/tensorflow/core/kernels/sparse_tensor_dense_matmul_op_gpu.cu.cc:20:
In file included from ./third_party/tensorflow/core/framework/bounds_check.h:21:
In file included from third_party/eigen3/Eigen/Core:155:
./third_party/eigen3/Eigen/src/Core/util/Meta.h:293:84: error: non-void function does not return a value [-Werror,-Wreturn-type]
  static T quiet_NaN() { assert(false && "quiet_NaN not supported for this type"); }
                                                                                   ^
./third_party/eigen3/Eigen/src/Core/NumTraits.h:208:39: note: in instantiation of member function 'Eigen::internal::device::numeric_limits<std::complex<double>>::quiet_NaN' requested here
    return numext::numeric_limits<T>::quiet_NaN();
                                      ^
third_party/tensorflow/core/kernels/sparse_tensor_dense_matmul_op_gpu.cu.cc:104:9: note: in instantiation of function template specialization 'tensorflow::SparseTensorDenseMatMulKernel<std::complex<double>, std::complex<double>, int, false, false>' requested here
        SparseTensorDenseMatMulKernel<T, Tsum, Tindices, ADJ_A, ADJ_B>,
        ^
In file included from third_party/tensorflow/core/kernels/sparse_tensor_dense_matmul_op_gpu.cu.cc:20:
In file included from ./third_party/tensorflow/core/framework/bounds_check.h:21:
In file included from third_party/eigen3/Eigen/Core:163:

I see the float and double specializations there, but nothing for complex types. Do you know where it's supposed to be coming from?

[benbarsdell]

I didn't realize there was no NumTraits implementation for complex (I guess compiler warning flags are more strict internally?).
I've pushed a fix for it.