Cleanup tensorflow/c/experimental/gradients Part 1

[vnghia]

@saxenasaurabh

One thing I don't understand, if computing numerical gradients with TensorFloat-32 is numerically unstable, does disabling TensorFloat-32 inside gradient_checker make a lot more sense than disabling it inside the test, since we will have to disable TF-32 in all binaries depending on gradient_checker anyway ?

[saxenasaurabh]

tensorflow/c/eager/unified_api_test.cc:124:76: error: invalid conversion from 'tensorflow::int64* {aka long long int*}' to 'int64_t* {aka long int*}' [-fpermissive]
TestTensorHandleWithDimsFloat(ctx.get(), data, dim_sizes, 2, &x_raw);

Please fix.

[vnghia]

tensorflow/c/eager/unified_api_test.cc:124:76: error: invalid conversion from 'tensorflow::int64* {aka long long int*}' to 'int64_t* {aka long int*}' [-fpermissive]
TestTensorHandleWithDimsFloat(ctx.get(), data, dim_sizes, 2, &x_raw);

Done

[rthadur]

@vnvo2409 can you please resolve conflicts ?

[vnghia]

@vnvo2409 can you please resolve conflicts ?

Done

[saxenasaurabh]

Looks like eager/BUILD has conflicts. Rebase maybe? Also nn_grad_test is segfaulting internally. I will try to patch and see what's going on.

[vnghia]

@saxenasaurabh

eager/BUILD has conflicts.

I've already fixed it.

Also nn_grad_test is segfaulting internally.

There are 3 memory problems with nn_grad_test.

---

One is comming from SoftMaxModel and RunAndMaybeSum

tensorflow/tensorflow/c/eager/gradient_checker.cc

Lines 68 to 73 in bd1fd58

	Status RunAndMaybeSum(AbstractContext* ctx, Model forward,
	absl::Span<AbstractTensorHandle* const> inputs,
	absl::Span<AbstractTensorHandle*> outputs,
	bool use_function) {
	GradientRegistry registry;
	std::vector<AbstractTensorHandle*> model_outputs(1);

SoftMaxModel returns 2 tensors but model_outputs take only 1 tensor so it causes a memory overflow. I intend to fix this issue in the next PR ( because running without asan is fine ), but we could fix in this PR if need though. WDYT ?

---

gradient_checker leaks a lot of AbstractTensorHandle. Will be fixed in the next PR too.

---

It seems that tape->ComputeGradients leaks a tensor. The traceback points to the allocation of a new tensor in BuildOnesLike. However, I could not find a fix for it yet.

[saxenasaurabh]

Ah got it. Let's fix the SoftmaxModel. The leaks we can address in a follow-up.

[vnghia]

Let's fix the SoftmaxModel.

Done

[vnghia]

@saxenasaurabh
The copybara/feedback failed