Ensure a prescribed threading layer can load in CI.

[stuartarchibald]

As title

[stuartarchibald]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[stuartarchibald]

Would an eval on the result of the look and followed by an assert in the string command be as clear as the additional "check result of query" in bash? Would save having to store the print into a bash var and test it later.

[sklam]

an uncaught exception should cause the python process to exit 1. Or you can also call os.exit(1).

[stuartarchibald]

Indeed, I'll give this a try to limit mixing two languages.

[sklam]

@stuartarchibald, do you want to make this change?

[stuartarchibald]

Done in 1b4666b. Now uses assert.

[sklam]

Thanks for the patch. I have a few comments.

[sklam]

an uncaught exception should cause the python process to exit 1. Or you can also call os.exit(1).

[sklam]

I find it hard to tell if this is running or not. Can use more echo when it passes

[stuartarchibald]

I've updated the python snippet to print on success in 1b4666b

[sklam]

Shouldn't this fail w/o #7605?

[stuartarchibald]

Shouldn't this fail w/o #7605?

I'm not entirely sure but I think not, #7605 adds both the check for the TBB version and also the correction to the library name. If this branch gains just the check for the TBB version I think it would fail, I'll try that now.

[stuartarchibald]

bcf1dc2 forces the TBB version check

[stuartarchibald]

Seemingly unrelated fail in linux, python 3.7, np1.18 with coverage:

2021-11-30T10:31:30.0079662Z INFO: Running slice of discovered tests: (3,None,19)

2021-11-30T10:35:40.6235501Z ======================================================================
2021-11-30T10:35:40.6236173Z ERROR: test_linspace_2 (numba.tests.test_dyn_array.TestLinspace)
2021-11-30T10:35:40.6237729Z ----------------------------------------------------------------------
2021-11-30T10:35:40.6238362Z Traceback (most recent call last):
2021-11-30T10:35:40.6238960Z   File "/home/vsts/work/1/s/numba/core/typeinfer.py", line 155, in propagate
2021-11-30T10:35:40.6239521Z     constraint(typeinfer)
2021-11-30T10:35:40.6240108Z   File "/home/vsts/work/1/s/numba/core/typeinfer.py", line 578, in __call__
2021-11-30T10:35:40.6240697Z     self.resolve(typeinfer, typevars, fnty)
2021-11-30T10:35:40.6241288Z   File "/home/vsts/work/1/s/numba/core/typeinfer.py", line 601, in resolve
2021-11-30T10:35:40.6241955Z     sig = typeinfer.resolve_call(fnty, pos_args, kw_args)
2021-11-30T10:35:40.6242705Z   File "/home/vsts/work/1/s/numba/core/typeinfer.py", line 1555, in resolve_call
2021-11-30T10:35:40.6243357Z     return self.context.resolve_function_type(fnty, pos_args, kw_args)
2021-11-30T10:35:40.6244009Z   File "/home/vsts/work/1/s/numba/core/typing/context.py", line 213, in resolve_function_type
2021-11-30T10:35:40.6244603Z     raise last_exception
2021-11-30T10:35:40.6245174Z   File "/home/vsts/work/1/s/numba/core/typing/context.py", line 196, in resolve_function_type
2021-11-30T10:35:40.6245830Z     res = self._resolve_user_function_type(func, args, kws)
2021-11-30T10:35:40.6246449Z   File "/home/vsts/work/1/s/numba/core/typing/context.py", line 248, in _resolve_user_function_type
2021-11-30T10:35:40.6247431Z     return func.get_call_type(self, args, kws)
2021-11-30T10:35:40.6248051Z   File "/home/vsts/work/1/s/numba/core/types/functions.py", line 331, in get_call_type
2021-11-30T10:35:40.6248630Z     failures.raise_error()
2021-11-30T10:35:40.6249207Z   File "/home/vsts/work/1/s/numba/core/types/functions.py", line 227, in raise_error
2021-11-30T10:35:40.6249808Z     raise errors.TypingError(self.format())
2021-11-30T10:35:40.6250507Z numba.core.errors.TypingError: No implementation of function Function(<function linspace at 0x7f07cf2e6f80>) found for signature:
2021-11-30T10:35:40.6251169Z  
2021-11-30T10:35:40.6251631Z  >>> linspace(int64, int64)
2021-11-30T10:35:40.6252078Z  
2021-11-30T10:35:40.6252575Z There are 4 candidate implementations:
2021-11-30T10:35:40.6253414Z       - Of which 4 did not match due to:
2021-11-30T10:35:40.6254396Z       Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 131.
2021-11-30T10:35:40.6255326Z         With argument(s): '(int64, int64)':
2021-11-30T10:35:40.6255953Z        Rejected as the implementation raised a specific error:
2021-11-30T10:35:40.6256662Z          TypingError: Failed in nopython mode pipeline (step: native lowering)
2021-11-30T10:35:40.6257331Z        Failed in nopython mode pipeline (step: nopython frontend)
2021-11-30T10:35:40.6258038Z        No implementation of function Function(<function linspace at 0x7f07cf2e6f80>) found for signature:
2021-11-30T10:35:40.6258662Z         
2021-11-30T10:35:40.6259155Z         >>> linspace(int64, int64, Literal[int](50))
2021-11-30T10:35:40.6259662Z         
2021-11-30T10:35:40.6260197Z        There are 4 candidate implementations:
2021-11-30T10:35:40.6261006Z              - Of which 4 did not match due to:
2021-11-30T10:35:40.6268797Z              Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 131.
2021-11-30T10:35:40.6269952Z                With argument(s): '(int64, int64, int64)':
2021-11-30T10:35:40.6270507Z               Rejected as the implementation raised a specific error:
2021-11-30T10:35:40.6271306Z                 TypingError: Failed in nopython mode pipeline (step: native lowering)
2021-11-30T10:35:40.6271877Z               Failed in nopython mode pipeline (step: nopython frontend)
2021-11-30T10:35:40.6272679Z               No conversion from array(float64, 1d, C) to array(float64, 1d, C) for '$40return_value.1', defined

Link to log: https://dev.azure.com/numba/ff1fe4d0-ed73-4f1c-b894-1d50a27e048f/_apis/build/builds/10403/logs/94

[stuartarchibald]

I can reproduce the above locally:

conda create -n _tmp_debug7600 -c numba python=3.7 numpy=1.18 gcc_linux-64 gxx_linux-64 gitpython scipy=1.6.2 llvmlite=0.38.0rc1 ipython jinja2 pygments pyyaml

Then install Numba merging this branch with mainline. Then do:

./runtests.py -b -j 3,None,19 --exclude-tags='long_running' -m 2 -- numba.tests

the hit rate for the error is unfortunately pretty low.

[stuartarchibald]

Cannot reproduce the issue on mailine (in 20 runs), I'd guess the problem on this branch is caused by some state from loading numba.np.ufunc.parallel, as that's the only material change to the python code in this patch.

[sklam]

There's now a conflict and a small comment:

[sklam]

Does this really need to be an identity check?

Suggested change

	assert get_sysinfo()['$1 Threading'] is True, 'Threading layer $1 '\
	assert get_sysinfo()['$1 Threading'], 'Threading layer $1 '\

[stuartarchibald]

I thought about this when writing it, I don't think there's any enforced return type for the output of get_sysinfo as it's largely informational only. My concern was that someone might patch this output to return a string like "It's working" or "It's not working", at which point a boolean assert would alias. If you feel differently am happy to change.

[sklam]

fair. we can leave it as is

[sklam]

Thanks for the patch!