Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sorting tests fail on Debian sid with numpy 1.26 #104

Open
avalentino opened this issue Mar 28, 2024 · 0 comments
Open

Sorting tests fail on Debian sid with numpy 1.26 #104

avalentino opened this issue Mar 28, 2024 · 0 comments

Comments

@avalentino
Copy link
Contributor

After the update of numpy to 1.26 we started observing the following failures in the compyle unittests both with Python 3.11. and 3.12.

The error is the following:

python3.12 -m pytest -k "not test_that_multiple_compiles_do_not_occur_for_same_source and not test_const_as_call_arg and not test_const_in_return" --ignore=/<<PKGBUILDDIR>>/compyle/tests/test_cuda.py --ignore=/<<PKGBUILDDIR>>/compyle/tests/test_parallel.py /<<PKGBUILDDIR>>/compyle/tests
> ============================= test session starts ==============================
> platform linux -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0
> rootdir: /<<PKGBUILDDIR>>
> configfile: pyproject.toml
> collected 327 items / 3 deselected / 324 selected
> 
> ../../../compyle/tests/test_array.py ..s..s..s..s..s..s..s..s..s..s..s.. [ 10%]
> s..s.FsF..s.sx..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s..s. [ 33%]
> .s..s..s..s..s..s..ss..s......sss......sss......sss..s                   [ 49%]
> ../../../compyle/tests/test_ast_utils.py .......                         [ 51%]
> ../../../compyle/tests/test_capture_stream.py .....                      [ 53%]
> ../../../compyle/tests/test_config.py ............                       [ 57%]
> ../../../compyle/tests/test_cython_generator.py .................        [ 62%]
> ../../../compyle/tests/test_ext_module.py ........                       [ 64%]
> ../../../compyle/tests/test_gpu_struct.py s                              [ 65%]
> ../../../compyle/tests/test_jit.py ..........................            [ 73%]
> ../../../compyle/tests/test_low_level.py s.s.....                        [ 75%]
> ../../../compyle/tests/test_profile.py .....                             [ 77%]
> ../../../compyle/tests/test_template.py .....                            [ 78%]
> ../../../compyle/tests/test_translator.py .............................. [ 87%]
> .........................                                                [ 95%]
> ../../../compyle/tests/test_transpiler.py ...                            [ 96%]
> ../../../compyle/tests/test_types.py .......                             [ 98%]
> ../../../compyle/tests/test_utils.py ....                                [100%]
> 
> =================================== FAILURES ===================================
> __________________________ test_sort_by_keys[opencl] ___________________________
> 
> backend = 'opencl'
> 
>     @check_all_backends
>     def test_sort_by_keys(backend):
>         check_import(backend)
>     
>         # Given
>         nparr1 = np.random.randint(0, 100, 16, dtype=np.int32)
>         nparr2 = np.random.randint(0, 100, 16, dtype=np.int32)
>         dev_array1, dev_array2 = array.wrap(nparr1, nparr2, backend=backend)
>     
>         # When
>         out_array1, out_array2 = array.sort_by_keys([dev_array1, dev_array2])
>     
>         # Then
>         order = np.argsort(nparr1)
>         act_result1 = np.take(nparr1, order)
>         act_result2 = np.take(nparr2, order)
>         assert np.all(out_array1.get() == act_result1)
> >       assert np.all(out_array2.get() == act_result2)
> E       assert False
> E        +  where False = <function all at 0x7f4cb39aa9f0>(array([ 9,  3...  dtype=int32) == array([ 9,  3...  dtype=int32)
> E        +    where <function all at 0x7f4cb39aa9f0> = np.all
> E           
> E           Use -v to get more diff)
> 
> ../../../compyle/tests/test_array.py:273: AssertionError
> ------------------------------ Captured log call -------------------------------
> DEBUG    pyopencl.cache:cache.py:378 build program: binary cache miss (key: 4bf4d189ce2047b8b06d9a0a5bca2e9d)
> DEBUG    pyopencl.cache:cache.py:413 build program: start building program from source on <pyopencl.Device 'cpu-skylake-avx512-Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz' on 'Portable Computing Language' at 0x2958b20>
> DEBUG    pyopencl.cache:cache.py:419 build program: from-source build complete
> DEBUG    pytools.persistent_dict:persistent_dict.py:758 pyopencl-invoker-cache-v41: disk cache hit [key=25d010a3d8d1b7c7137e612326d8db5d04a1e8ec5bb2a48c57295ed6430bda96]
> INFO     pyopencl:__init__.py:469 build program: kernel 'get_size_and_offsets' was part of a lengthy source build resulting from a binary cache miss (0.28 s)
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-generated-scan-kernel-cache-v1: disk cache miss [key=611aa02c9ab60fec70c25bc297d9b1653da0a6e28c0f8ca21971a6c5efbcd16e]
> DEBUG    pyopencl.scan:scan.py:1194 cache miss for generated scan kernel 'scan'
> DEBUG    pyopencl.cache:cache.py:378 build program: binary cache miss (key: 653143575044338b788fc83353afc3f7)
> DEBUG    pyopencl.cache:cache.py:413 build program: start building program from source on <pyopencl.Device 'cpu-skylake-avx512-Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz' on 'Portable Computing Language' at 0x2958b20>
> DEBUG    pyopencl.cache:cache.py:419 build program: from-source build complete
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-invoker-cache-v41: disk cache miss [key=49efc131ac55675366d11c67a6021ffc9ab51061e766db708184340ec1e7b944]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-invoker-cache-v41: disk cache store [key=49efc131ac55675366d11c67a6021ffc9ab51061e766db708184340ec1e7b944]
> INFO     pyopencl:__init__.py:469 build program: kernel 'scan_lev1' was part of a lengthy source build resulting from a binary cache miss (1.37 s)
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-invoker-cache-v41: disk cache miss [key=eca878c5a15faa094d2cd87ee42508762ce2d70071b18e16df9b2f56071b9e2a]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-invoker-cache-v41: disk cache store [key=eca878c5a15faa094d2cd87ee42508762ce2d70071b18e16df9b2f56071b9e2a]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-generated-scan-kernel-cache-v1: disk cache store [key=611aa02c9ab60fec70c25bc297d9b1653da0a6e28c0f8ca21971a6c5efbcd16e]
> DEBUG    pyopencl.cache:cache.py:384 build program: binary cache hit (key: 653143575044338b788fc83353afc3f7)
> DEBUG    pytools.persistent_dict:persistent_dict.py:758 pyopencl-invoker-cache-v41: disk cache hit [key=49efc131ac55675366d11c67a6021ffc9ab51061e766db708184340ec1e7b944]
> DEBUG    pytools.persistent_dict:persistent_dict.py:758 pyopencl-invoker-cache-v41: disk cache hit [key=eca878c5a15faa094d2cd87ee42508762ce2d70071b18e16df9b2f56071b9e2a]
> DEBUG    pyopencl.cache:cache.py:378 build program: binary cache miss (key: a7aa988ad08d4c5636b571b0cd4ae55e)
> DEBUG    pyopencl.cache:cache.py:413 build program: start building program from source on <pyopencl.Device 'cpu-skylake-avx512-Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz' on 'Portable Computing Language' at 0x2958b20>
> DEBUG    pyopencl.cache:cache.py:419 build program: from-source build complete
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-invoker-cache-v41: disk cache miss [key=02d42ae23a40f1caecefc2770cde1b1b442262a489c1b3a002673425df9233b7]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-invoker-cache-v41: disk cache store [key=02d42ae23a40f1caecefc2770cde1b1b442262a489c1b3a002673425df9233b7]
> INFO     pyopencl:__init__.py:469 build program: kernel 'scan_lev2' was part of a lengthy source build resulting from a binary cache miss (1.30 s)
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-invoker-cache-v41: disk cache miss [key=d913f3cd98e8844fab3f92bfc28d0e5dca7af2f7da3b784402ba106d08be4e68]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-invoker-cache-v41: disk cache store [key=d913f3cd98e8844fab3f92bfc28d0e5dca7af2f7da3b784402ba106d08be4e68]
> DEBUG    pyopencl.cache:cache.py:378 build program: binary cache miss (key: fdd1b09b4764e411254319832d21c5fb)
> DEBUG    pyopencl.cache:cache.py:413 build program: start building program from source on <pyopencl.Device 'cpu-skylake-avx512-Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz' on 'Portable Computing Language' at 0x2958b20>
> DEBUG    pyopencl.cache:cache.py:419 build program: from-source build complete
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-invoker-cache-v41: disk cache miss [key=01951cb5ce3fe8984bb56f98fa6e9163f6396390a612759f911c18d9a47df5c2]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-invoker-cache-v41: disk cache store [key=01951cb5ce3fe8984bb56f98fa6e9163f6396390a612759f911c18d9a47df5c2]
> INFO     pyopencl:__init__.py:469 build program: kernel 'scan_final_update' was part of a lengthy source build resulting from a binary cache miss (0.29 s)
> DEBUG    pytools.persistent_dict:persistent_dict.py:727 pyopencl-invoker-cache-v41: disk cache miss [key=f41f4aa10a5614c18e468c40b4ca31d6c64a59c262ed220e22930319f10ea84b]
> DEBUG    pytools.persistent_dict:persistent_dict.py:696 pyopencl-invoker-cache-v41: disk cache store [key=f41f4aa10a5614c18e468c40b4ca31d6c64a59c262ed220e22930319f10ea84b]
> ___________________________ test_radix_sort_by_keys ____________________________
> 
>     def test_radix_sort_by_keys():
>         backend = 'cython'
>         for use_openmp in [True, False]:
>             get_config().use_openmp = use_openmp
>             # Given
>             nparr1 = np.random.randint(0, 100, 16, dtype=np.int32)
>             nparr2 = np.random.randint(0, 100, 16, dtype=np.int32)
>             dev_array1, dev_array2 = array.wrap(nparr1, nparr2, backend=backend)
>     
>             # When
>             out_array1, out_array2 = array.sort_by_keys([dev_array1, dev_array2],
>                                                         use_radix_sort=True)
>     
>             # Then
>             order = np.argsort(nparr1)
>             act_result1 = np.take(nparr1, order)
>             act_result2 = np.take(nparr2, order)
>             assert np.all(out_array1.get() == act_result1)
> >           assert np.all(out_array2.get() == act_result2)
> E           assert False
> E            +  where False = <function all at 0x7f4cb39aa9f0>(array([33,  7...  dtype=int32) == array([33,  7...  dtype=int32)
> E            +    where <function all at 0x7f4cb39aa9f0> = np.all
> E               
> E               Use -v to get more diff)
> 
> ../../../compyle/tests/test_array.py:294: AssertionError
> ----------------------------- Captured stdout call -----------------------------
> [1/1] Cythonizing /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/m_38c1c650fcf31d53befb93aa70eceb48.pyx
> ------------------------------ Captured log call -------------------------------
> INFO     compyle.ext_module:ext_module.py:321 Compiling code at: /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/m_38c1c650fcf31d53befb93aa70eceb48.pyx
> DEBUG    root:discovery.py:490 No `name` configuration, performing automatic discovery
> INFO     root:dist.py:985 running build_ext
> INFO     root:build_ext.py:521 building 'm_38c1c650fcf31d53befb93aa70eceb48' extension
> INFO     root:spawn.py:38 x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/python3/dist-packages/numpy/core/include -I/usr/include/python3.12 -c /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/m_38c1c650fcf31d53befb93aa70eceb48.cpp -o /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/build/temp.linux-x86_64-cpython-312/<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/m_38c1c650fcf31d53befb93aa70eceb48.o -fopenmp
> INFO     root:spawn.py:38 x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wdate-time -D_FORTIFY_SOURCE=2 /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/build/temp.linux-x86_64-cpython-312/<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/m_38c1c650fcf31d53befb93aa70eceb48.o -L/usr/lib/x86_64-linux-gnu -o /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_compyle/.compyle/source/py3.1-linux-x86_64/build/lib.linux-x86_64-cpython-312/m_38c1c650fcf31d53befb93aa70eceb48.cpython-312-x86_64-linux-gnu.so -fopenmp
> =============================== warnings summary ===============================
> compyle/tests/test_array.py: 6 warnings
> compyle/tests/test_jit.py: 5 warnings
> compyle/tests/test_translator.py: 93 warnings
>   /usr/lib/python3.12/ast.py:407: DeprecationWarning: visit_Num is deprecated; add visit_Constant
>     return visitor(node)
> 
> compyle/tests/test_array.py: 4 warnings
> compyle/tests/test_translator.py: 93 warnings
>   /<<PKGBUILDDIR>>/compyle/translator.py:682: DeprecationWarning: Attribute n is deprecated and will be removed in Python 3.14; use value instead
>     return str(node.n)

[CUT]

> =========================== short test summary info ============================
> FAILED ../../../compyle/tests/test_array.py::test_sort_by_keys[opencl] - asse...
> FAILED ../../../compyle/tests/test_array.py::test_radix_sort_by_keys - assert...
> = 2 failed, 264 passed, 57 skipped, 3 deselected, 1 xfailed, 354 warnings in 334.14s (0:05:34) =
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant