TST, BLD: Fix failing aarch64 wheel builds.

[charris]

The aarch64 wheel build tests are failing with OOM. The new test for complex128 dot for huge vectors is responsible as the usable memory is incorrectly determined and the check for sufficient memory fails. The fix here is to set NPY_AVAILABLE_MEM="4 GB" in the environment before calling the test in cibw_test_command.sh.

[charris]

Hmm, the aarch64 wheel builds are reported as passing, but they are actually failing.

[mattip]

Which test? Does it specify a skipif(memory requirement)?

[charris]

Which test?

Trying to figure that out. The one I currently suspect does use memory requirement, but I am not sure of it yet. For some reason sys.exit(not numpy.test('full')) reports success following OOM, but that is a separate problem.

[charris]

Looks like test_huge_vectordot. If I unconditionally skip it, the tests run to completion.

    @pytest.mark.slow
    @pytest.mark.parametrize("dtype", [np.float64, np.complex128])
    @requires_memory(free_bytes=18e9)  # complex case needs 18GiB+
    def test_huge_vectordot(self, dtype):
        # Large vector multiplications are chunked with 32bit BLAS
        # Test that the chunking does the right thing, see also gh-22262
        pytest.skip('testing')
        data = np.ones(2**30+100, dtype=dtype)
        res = np.dot(data, data)
        assert res == 2**30+100

May be a problem with memory detection on aarch64.

[seberg]

Hmmm, agree that it looks like I estimated it for 32bit floats and then also added 64bit or so. But strange that it requires the unconditional skip. OTOH not sure it is worth to dig deep, it seems also likely to me that the available memory detection is simply not be reliable on aarch.

[charris]

@seberg Your estimate was low, but even with that fixed it still goes OOM. For the 1.23.4 release I will probably skip it unconditionally, but I want to fool with it a bit on main to see what is going wrong.

[charris]

OK, here is the problem:

mem is 129400066048, platform is aarch64

That's 120 GB. I wonder if we should try getting the allowed memory

[mattip]

I could not find the machine information in the travis documentation in the ten minutes I tried. Would it be easy to add free -h to the bash output somewhere?

[charris]

@mattip I think the machine the test is running on does have 120 GB of memory, but the memory allowed to the process is much less. I also wonder why we don't get memory_error instead of OOM. There is a resource module in the standard library that might help, but on my machine it simply returns (-1, -1) for data memory, which apparently means infinite :) Another option would be to set the NPY_AVAILABLE_MEM environment variable in cibcibw_test_command.sh before the test is run. That might be a better way to do this.

[charris]

Trying the environment variable approach.

[charris]

Self merging so I can backport and do the 1.23.4 release.