Skip to content

Conversation

@ngoldbaum
Copy link
Member

@ngoldbaum ngoldbaum commented Mar 12, 2024

This is an attempt to fix #25875.

The tests are hanging, not crashing, so the behavior looks like a deadlock. This is the only use of ProcessPoolExecutor in the numpy tests and the tests seem to hang on this test.

That said, I'm not sure why the deadlock would only happen in a build of python running under address sanitizer.

Looking at the test again, I don't think this is actually a useful thing to test, since it shouldn't matter if the pickle file is loaded in the process that saved it or not.

So since I suspect this will fix the test hangs, I think this is worth pulling in.

@ngoldbaum
Copy link
Member Author

Also for context, here's the output from a build that hangs, along with timestamps:

Sat, 09 Mar 2024 21:19:24 GMT numpy/_core/tests/test_stringdtype.py::test_isnan[string nan-True] PASSED
Sat, 09 Mar 2024 21:19:24 GMT numpy/_core/tests/test_stringdtype.py::test_isnan[string nan-False] PASSED
Sat, 09 Mar 2024 21:19:24 GMT numpy/_core/tests/test_stringdtype.py::test_pickle[unset-True] PASSED
Sat, 09 Mar 2024 21:19:24 GMT numpy/_core/tests/test_stringdtype.py::test_pickle[unset-False] PASSED
Sat, 09 Mar 2024 21:19:24 GMT numpy/_core/tests/test_stringdtype.py::test_pickle[None-True] PASSED
Sun, 10 Mar 2024 03:03:08 GMT

Sun, 10 Mar 2024 03:03:10 GMT Session terminated, killing shell... ...killed.
Sun, 10 Mar 2024 03:03:10 GMT numpy/_core/tests/test_stringdtype.py::test_pickle[None-False] 
Sun, 10 Mar 2024 03:03:10 GMT Error: The operation was canceled.

Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @ngoldbaum. I agree that it's not useful to test with ProcessPoolExecutor, and tests like these can cause problems. So +1 for merging even if it doesn't fix the ASan job.

@mattip
Copy link
Member

mattip commented Mar 13, 2024

+1 for merging even if it doesn't fix the ASan job.

The ASan job was canceled after 236 minutes and the log is unavailable. But I will merge anyway. Thanks @ngoldbaum

@mattip mattip merged commit ec3b744 into numpy:main Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: Compiler sanitizers tests are hanging intermittently

3 participants