PERF: releasing the GIL, #8882

[jreback]

closes #8882
closes #10139

This is now implemented for all groupbys with the factorization part as well (which is actually a big part fo the time)

In [2]: N = 1000000

In [3]: ngroups = 1000

In [4]: np.random.seed(1234)

In [5]: df = DataFrame({'key' : np.random.randint(0,ngroups,size=N),
   ...:                 'data' : np.random.randn(N) })

def f():
    df.groupby('key')['data'].sum()

# run consecutivily
def g2():
    for i in range(2):
        f()
def g4():
    for i in range(4):
        f()
def g8():
    for i in range(8):
        f()

# run in parallel
@test_parallel(num_threads=2)
def pg2():
    f()

@test_parallel(num_threads=4)
def pg4():
    f()

@test_parallel(num_threads=8)
def pg8():
    f()

So seeing a nice curve as far as addtl cores are used (as compared to some of my previous posts).

In [19]: df
Out[19]: 
   after  before   speedup
2   27.3    52.1  1.908425
4   44.8   105.0  2.343750
8   83.2   213.0  2.560096

[mrocklin]

What was the deal with NA and nan. These were Python objects?

[jreback]

apparently doing something

cdef ndarray[double_t] foo

foo = np.empty(10,dtype='float64')

foo[0] = np.nan

actually assigns a python object (and does inference and such), so that you MUST have the gil,
but casting to a double seems to work though (as its now c-level)

cdef double nan = <double> np.nan
with nogil:
    foo[0] = nan

[ahojnnes]

You can reuse NPY_NAN cross-platform, e.g., https://github.com/scikit-image/scikit-image/blob/master/skimage/feature/_texture.pyx#L11

[jreback]

@ahojnnes thxs, I did change it.

[jreback]

I updated this. works for all groupbys now. So it gives a general 5-10% speedup on most operations (even single threaded), as there is slightly less generated code (in nogil it pretty much generates straight c code).

so we still need to hold the gil for the .resize operation, but its generally quite limited when this happens (e.g. resizing is a function of the number of uniques for example), but the cost of this is pretty low.

There IS a soln where I can not use numpy resize at ALL. But its quite a bit more c-code and IMHO not really worth it.

[mrocklin]

Cool. I'll try to run a few different groupby operations with dask.dataframe sometime early next week and report back. I'm curious what were the operations that needed to be released? Was there a core set of functionality or was it a large volume of trivial work?

[jreback]

TL;DR.

So lots of 'boilerplate' changes in generated.pyx, which is basically all of the algorithms for groupby ops themselves (e.g. sum,mean). These were straightforward insertions of with nogil, just needed to avoid typing the object ones (which for the most part was easy; though a bit tricky on some non-groupby routines, e.g. see #10213).

However, the big speeds occur once I fixed the hashtable.pyx factorizer (e.g. the get_labels routine). The trivial soln actually produces a very odd perf issue (see here), which is a bug / documentation issue.

In effect, you are using a nogil function that happens to have a with gil block WITHIN it, then the function is way slower than if you inline the exact same code.

I 'fixed' this issue by adding some additional c-structures to hold data (basically I changed the low-level implementation); these can be easily passed around; downside is that are a little 'messy' in the code department.

note to @shoyer I didn't actually bypass the realloc of memory (e.g. .resize) of numpy arrays. though I do have a soln to that (basically manage a memory block myself, then turning that into a numpy array at the very end (in to_array), easy enough, but was running into some compile issues, so dropped it in the end).

[mrocklin]

I think that this is awesome and am afraid of it going stale. Is there something blocking it from being merged into master?

[shoyer]

@mrocklin We were waiting to get the 0.16.2 release out first, which happened last Friday. At this point I don't think there are any blockers.

[jreback]

nothing blocking. I am going to update and see if I can hit some of #10213.

I have to do a bit of odd code-generation because for example you want to simultaneously support nogil (for numeric) and gil for object, but its a context manager (and NOT a function, which has some odd per characteristics). IOW, you have to worry about formatting, a bit annoying.

[jreback]

actually I do need to add a doc-note to the whatsnew (and maybe a small mention in the enhancingperf section as well).

[mrocklin]

Hrm, yes, unfortunate that you can't manipulate the lock directly

[mrocklin]

Thought I'd caress this PR with a gentle ping.

ping

[jreback]

is that euphemism for harassment?

[mrocklin]

Kind and loving harassment, yes.

[jreback]

any commentary @jorisvandenbossche @shoyer @cpcloud @TomAugspurger

going to have a followup for more GIL stuff. Its actually tricky making sure its doing (and you are measure exactly the right stuff)

[cpcloud]

i'd say "benefit from this"

[jreback]

@cpcloud I got this to work after puzzling on what cython does with multiple fused types. It creates the cross-product, but you generally need only certain specilizations; it does this beaut as a compile-time prune, so no perf hit.

[sinhrks]

Great job!!

[mrocklin]

Woohoo!

Pulls from master, checks benchmarks.

[jorisvandenbossche]

@mrocklin Can we already look forward to a new awesome blogpost? :-)

[mrocklin]

Best I have to offer at the moment are the slowly growing dask.dataframe docs

[jorisvandenbossche]

I should definitely check it out once I find some time!

[scari]

Wow! Great job! 👍🏻