Support nogil=True for ufuncs and gufuncs

[shoyer]

In [3]: @numba.vectorize(nogil=True)
   ...: def f(a, b):
   ...:     return a + b
   ...:

In [4]: f(1, 2)
KeyError: "Does not support option: 'nogil'"

This would be extremely useful for integrating numba ufuncs with dask.array.

[sklam]

I'm able to use all 4 cores with the following example. Numba is using numpy ufunc machinery, which is releasing the GIL. Note that it may need to warm up because the function is compiled on the first call.

from numba import vectorize
import numpy as np

@vectorize
def f(a, b):
    return np.sqrt(a) + np.cos(b) * np.sin(b) - a


nelem = 10**8
a = np.arange(nelem)
z = f(a, a)
print(z)


from concurrent.futures import ThreadPoolExecutor, as_completed

numthreads = 4
pool= ThreadPoolExecutor(numthreads)

futures = [pool.submit(f, a, a) for i in range(numthreads)]
for z in as_completed(futures):
    print(z)

[seibert]

In this case, should we make nogil=True on @vectorize simply print a warning that ufuncs always release the GIL?

[shoyer]

I'm pleased to see this already exists!

I agree, a warning if nogil=True is passed seems appropriate.

[stuartarchibald]

Adding a warning for nogil=True being passed is the good first issue.

[sklam]

The enhancement task is: #1317 (comment)

[pradkrish]

is this task still available? I am a newcomer here and I would love to get my feet wet(with some guidance). Thanks.

[pradkrish]

In numba/np/ufunc/ufuncbuilder.py:85, before calling the method parse_as_flags(..), is that a good place to add a warning if nogil=True is present as one of the keys in topt?

[stuartarchibald]

is this task still available? I am a newcomer here and I would love to get my feet wet(with some guidance). Thanks.

Yes! Thanks for taking it on.

[stuartarchibald]

In numba/np/ufunc/ufuncbuilder.py:85, before calling the method parse_as_flags(..), is that a good place to add a warning if nogil=True is present as one of the keys in topt?

That looks like a good place as it'll also catch the use in guvectorize.

[pradkrish]

Lovely. Another question: After printing the warning, do you think nogil=True key-value pair should be deleted from topt before passing it on so that there won't be any KeyError down the chain?