Continuation of PR #7280, fixing lifetime of TBB task_scheduler_handle

[stuartarchibald]

As title.

[stuartarchibald]

96edef4 fails for the TBB build with https://dev.azure.com/numba/numba/_build/results?buildId=9489&view=logs&j=064bd7ca-23e4-5078-9cb6-8968eb13bf19&t=30a4dffd-4ede-53d2-ca39-79d23392ba79&l=409 (segfault)

======================================================================
FAIL: test_lifetime_of_task_scheduler_handle (numba.tests.test_parallel_backend.TestTBBSpecificIssues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vsts/work/1/s/numba/tests/test_parallel_backend.py", line 1032, in test_lifetime_of_task_scheduler_handle
    out, err = self.run_cmd(cmdline, env=env)
  File "/home/vsts/work/1/s/numba/tests/test_parallel_backend.py", line 478, in run_cmd
    (popen.returncode, err.decode()))
AssertionError: process failed with code -11: stderr follows
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
Fatal Python error: Segmentation fault

Current thread 0x00007f8487c04740 (most recent call first):
Fatal Python error: Segmentation fault

Current thread 0x00007f8487c04740 (most recent call first):

[stuartarchibald]

792e2d4 contains @PokhodenkoSA / @Hardcode84 patch for the issue.

[stuartarchibald]

792e2d4 failed once with :

Traceback (most recent call last):
  File "/home/vsts/miniconda3/envs/azure_ci/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/vsts/miniconda3/envs/azure_ci/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/vsts/work/1/s/numba/runtests.py", line 9, in <module>
    sys.exit(0 if _main(sys.argv) else 1)
  File "/home/vsts/work/1/s/numba/testing/_runtests.py", line 26, in _main
    **kwds).wasSuccessful()
  File "/home/vsts/work/1/s/numba/testing/__init__.py", line 60, in run_tests
    nomultiproc=nomultiproc)
  File "/home/vsts/work/1/s/numba/testing/main.py", line 168, in __init__
    super(NumbaTestProgram, self).__init__(*args, **kwargs)
  File "/home/vsts/miniconda3/envs/azure_ci/lib/python3.7/unittest/main.py", line 101, in __init__
    self.runTests()
  File "/home/vsts/work/1/s/numba/testing/main.py", line 340, in runTests
    run_tests_real()
  File "/home/vsts/work/1/s/numba/testing/main.py", line 325, in run_tests_real
    super(NumbaTestProgram, self).runTests()
  File "/home/vsts/miniconda3/envs/azure_ci/lib/python3.7/unittest/main.py", line 271, in runTests
    self.result = testRunner.run(self.test)
  File "/home/vsts/work/1/s/numba/testing/main.py", line 795, in run
    return super(ParallelTestRunner, self).run(self._run_inner)
  File "/home/vsts/miniconda3/envs/azure_ci/lib/python3.7/unittest/runner.py", line 176, in run
    test(result)
  File "/home/vsts/work/1/s/numba/testing/main.py", line 743, in _run_inner
    self._run_parallel_tests(result, pool, child_runner, tests)
  File "/home/vsts/work/1/s/numba/testing/main.py", line 779, in _run_parallel_tests
    raise e
  File "/home/vsts/work/1/s/numba/testing/main.py", line 770, in _run_parallel_tests
    child_result = it.__next__(self.timeout)
  File "/home/vsts/miniconda3/envs/azure_ci/lib/python3.7/multiprocessing/pool.py", line 743, in next
    raise TimeoutError from None
multiprocessing.context.TimeoutError: Tests didn't finish before timeout (or crashed):
- 'numba.tests.test_parallel_backend.TestSpecificBackend.test_random_concurrent_mix_use_masks_tbb'

[sklam]

Thanks for the patch. I have a few comments.

[sklam]

I checked that these changes are needed to make the new test pass.

[stuartarchibald]

Great, thanks.

[sklam]

should this return a different exit code?

[stuartarchibald]

The self.run_cmd checks for exit code 0 and raises assertion error if it is not. What I was attempting to do here is differentiate between "This test cannot be run because of some compiler setup problem" and "The compiler setup was fine and the test can be run, but is has failed". The test uses a non-zero exit code which results in the assertion error to signal the latter and a zero exit code with a special token assigned to the BROKEN_COMPILERS variable for the former. Should the BROKEN_COMPILERS condition occur, the test sets a self.skipTest().

[sklam]

As agreed in OOB conversion, there are too many ways for the compilation to fail. We will allow the test to be a little fragile (may accidentally skip). Later we will look into ways to make sure the test is not skipped on systems that we have tight control and can guarantee the test must work.

[stuartarchibald]

This needs to go through the build farm a couple of times to see if the test time out happens on machines with more resources.

[stuartarchibald]

Buildfarm ID: numba_smoketest_cpu_yaml_34.

[sklam]

Thanks for the patch!

The test maybe fragile but it is not worth stressing over accidentally skipping it for this rare problem.