parallel list comprehension corruption

[stuartarchibald]

Reporting a bug

• I am using the latest released master version of Numba (most recent is visible in
  the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included below a minimal working reproducer (if you are unsure how
  to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

This:

from numba import njit, prange, gdb
import numpy as np

@njit(parallel=True)
def get_lengths_parallel(xx):
    return [np.int32(len(xx[i])) for i in prange(len(xx))]

get_lengths_parallel([np.array([1,2,3],dtype=np.int64) for i in range(1000000)])

is corrupting state somewhere, sometimes SIGSEGV, sometimes double free and SIGABRT, sometimes realloc(): invalid next size and SIGSEGV. Issue seems to be invariant of threading layer.

[mroeschke]

Just wanted to note that this bug still exists in 0.50.1 and there might be subtle (but related) differences when using list comprehension vs list.append

In [5]: import numpy as np

In [6]: import numba

In [7]: numba.__version__
Out[7]: '0.50.1'

In [3]: @numba.jit(nopython=True, parallel=True)
   ...: def f(x):
   ...:     return [np.arange(i) for i in numba.prange(x)]

In [4]: f(4)
python(6352,0x700001bec000) malloc: Double free of object 0x7fbe6a494110
python(6352,0x700001c6f000) malloc: Double free of object 0x7fbe6a494110
python(6352,0x700001bec000) malloc: *** set a breakpoint in malloc_error_break to debug
python(6352,0x700001c6f000) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6

In [3]: @numba.jit(nopython=True, parallel=True)
   ...: def f(x):
   ...:     lst = []
   ...:     for i in numba.prange(x):
   ...:         lst.append(np.arange(i))
   ...:     return lst
   ...:

In [4]: f(4)
python(6370,0x70000d504000) malloc: *** error for object 0x7febad438f20: pointer being realloc'd was not allocated
python(6370,0x70000d504000) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6

[esc]

@mroeschke thanks for updating the issue and adding more information.

[stuartarchibald]

Closing, this is unsupported as containers such as list as not threadsafe. Noted in docs:
https://numba.readthedocs.io/en/stable/user/parallel.html#unsupported-operations