make torch.testing asserts importable

[pmeier]

Stack from ghstack:

• Add support for checking tensor containers in torch.testing #55385 Add support for checking tensor containers in torch.testing
• make torch.testing error messages more expressive #55145 make torch.testing error messages more expressive
• add tests for torch.testing #54784 add tests for torch.testing
• torch.testing allclose -> close #54781 torch.testing allclose -> close
• change torch.testing helper asserts to checks #54780 change torch.testing helper asserts to checks
• make torch.testing asserts importable #54769 make torch.testing asserts importable

Follow-up to #53820. This

• makes the asserts.py module private as per suggestion from @rgommers in initial draft for assert_tensors_(equal|allclose) in torch.testing #53820 (comment). With this the functions should only be accessible through torch.testing, giving us the option the change the underlying structure later.
• moves the code from torch/testing/__init__.py to torch/testing/_core.py (happy to accept other name suggestions). Otherwise we can't import the new _asserts.py in torch/testing/__init__.py due to circular imports.
• factors out some functionality from torch/testing/_internal/common_utils.py that is needed in new _asserts.py to torch/testing/_internal/testing_utils.py to not be dependent on numpy at import.

Differential Revision: D27438451

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 436356e (more details on the Dr. CI page):

---

💚 💚 Looks good so far! There are no failures yet. 💚 💚

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[pmeier]

@mruberry Not sure about the test failures. For some reason importing numpy is failing. From the error message

ModuleNotFoundError: Sorry PyTorch, but our NumPy is in the other folder

it seems someone planned for this. Do you have insights?

[mruberry]

@mruberry Not sure about the test failures. For some reason importing numpy is failing. From the error message

ModuleNotFoundError: Sorry PyTorch, but our NumPy is in the other folder

it seems someone planned for this. Do you have insights?

I'd spend some more time debugging (possibly after a rebase for good measure). It looks like two tests are failing. The ModuleNotFoundError is coming from a "fake NumPy" that was added for testing purposes in February. What's causing it to fail?

[pmeier]

#52794 added a test to check that torch is importable

if PyTorch was compiled with wrong NumPy version, of NumPy is not commonly available on the platform (which is often the case on AARCH64 or Apple M1)

The new _asserts.py needs functionality from torch.testing._internal which imports numpy unconditionally. Since this has not happened before, the tests went fine. I'll factor out the functionality that we need into a separate file.

[mruberry]

Cool; so this also fixes that importing torch imports numpy?

I wonder if we should add a test for that?

[mruberry]

#52794 added a test to check that torch is importable

if PyTorch was compiled with wrong NumPy version, of NumPy is not commonly available on the platform (which is often the case on AARCH64 or Apple M1)

The new _asserts.py needs functionality from torch.testing._internal which imports numpy unconditionally. Since this has not happened before, the tests went fine. I'll factor out the functionality that we need into a separate file.

I like the comment you made in the code about this. Would you update this PR description, too? We should talk about our plan for the structure of all this code at our next testing sync next week.

[pmeier]

Cool; so this also fixes that importing torch imports numpy?

Correct.

I wonder if we should add a test for that?

The test that failed before tests for exactly that:

pytorch/.jenkins/pytorch/test.sh

Lines 160 to 164 in 444e5f0

	test_without_numpy() {
	pushd "$(dirname "${BASH_SOURCE[0]}")"
	python -c "import sys;sys.path.insert(0, 'fake_numpy');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())"
	popd
	}

Still, the error message through me off

pytorch/.jenkins/pytorch/fake_numpy/numpy.py

Line 1 in 444e5f0

raise ModuleNotFoundError("Sorry PyTorch, but our NumPy is in the other folder")

Maybe we can change it into something more descriptive like

raise ModuleNotFoundError("Importing 'torch' must not require 'numpy' to be present.")

Cc @malfet

[mruberry]

I need to investigate this further internally. The new structure either requires some internal tweaks or there's a problem with the specifying the imports relative to the current location.

Example failure:

  File ".../tree/torch/testing/_asserts.py", line 7, in <module>
    from ._internal.testing_utils import (
ModuleNotFoundError: No module named 'torch.testing._internal'

[mruberry]

I had some more time to investigate the internal build system, and I think the issue is simply that the current build system isn't designed to allow _internal to be imported into testing, since torch.testing is always imported but torch.testing._internal is only imported on test builds.

We could try and tweak the internal build system, but I'd prefer we just work around the issue. Can we just copy parts of _internal we need for asserts.py into torch/testing/asserts.py for now, for example?

[pmeier]

Can we just copy parts of _internal we need for asserts.py into torch/testing/asserts.py for now, for example?

Yes, we can. I've removed the torch/testing/_internal/testing_utils.py again and simply copied the dtype precisions. Note that copying get_comparison_dtype was not necessary, since the todos are outdated.

pytorch/torch/testing/_internal/common_utils.py

Lines 858 to 873 in df299db

	def get_comparison_dtype(a, b):
	# TODO: update this when promote_types supports bfloat16 and/or
	# isclose supports bfloat16.
	a_dtype = torch.float32 if a.dtype is torch.bfloat16 else a.dtype
	b_dtype = torch.float32 if b.dtype is torch.bfloat16 else b.dtype
	compare_dtype = torch.promote_types(a_dtype, b_dtype)
	# non-CUDA (CPU, for example) float16 -> float32
	# TODO: update this when isclose is implemented for CPU float16
	if (compare_dtype is torch.float16 and
	(a.device != b.device or a.device.type != 'cuda' or
	b.device.type != 'cuda')):
	compare_dtype = torch.float32
	return compare_dtype

Thus, it can simply replaced by torch.promote_types().

[mruberry]

Let's see what the internal build system thinks.

[facebook-github-bot]

@mruberry merged this pull request in f4967d6.