Adding and use `assertAllClose` API in our tests.

[ailzhang]

Currently we have many uses of

self.assertEqual(ans.shape, expected.shape)
self.assertTrue(np.allclose(ans.cpu(), expected, rtol=1e-02, atol=1e-03, equal_nan=True))

in our tests. e.g. https://github.com/pytorch/pytorch/blob/master/test/test_torch.py#L11270
torch.allclose won't work well here as it only accept two tensors.
But this way is hard for backends like XLA to override precisions :P
We want to propose make an API

self.assertAllClose(torch_tensor, np_array, rtol, atol, equal_nan)

to replace it.
@mruberry @gchanan @ezyang Does this sound good?
cc: @JackCaoG

[ezyang]

Existing assertEqual has tolerance kwargs; is this not working for the xla case for some reason?

[ailzhang]

@ezyang assertEqual works since XLA can override the precision.
But when PyTorch test calls np.allclose instead of assertEqual(e.g. https://github.com/pytorch/pytorch/blob/master/test/test_torch.py#L11270) , it'll be tricky for XLA to override precision. ;)

[gchanan]

it sounds like we should just not use np.allclose.

[ailzhang]

@gchanan Yup but we do need an allclose-like API since it doesn't check dtype while assertEqual might/will?

[ezyang]

assertEqual currently doesn't check dtype, and even after my tightening up, you can expliictly request no dtype check with exact_dtype=False

[ailzhang]

Nice, sounds like an easier solution is just replace all uses of np.allclose in our tests with assertEqual(..., exact_dtype=False)?
I'll send a PR if that sounds good!

[ezyang]

Yeah, that would be great. (Note that most tests still default to exact_dtype=False, but we want to fix that.)

[ailzhang]

Oh I just realized we do have a assert_allclose API already. But it's not in TestCase class. https://github.com/pytorch/pytorch/blob/master/torch/testing/__init__.py#L18
I somehow feel they should be in one place... Let me know if you have any preference between it and assertEqual. ;)

[ezyang]

Yes, we should put them all in one place :> TestCase seems like the logical spot.

[ailzhang]

I feel @nairbv 's PR #34258 is bringing closer by adding atol and rtol to assertEqual, I'll hold my PR until his is merged.