issue with prange writing to preallocated list entries

[stuartarchibald]

Reporting a bug

• I am using the latest released master version of Numba (most recent is visible in
  the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included below a minimal working reproducer (if you are unsure how
  to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).
  This:

from numba import njit, prange
import numpy as np

@njit(parallel=True)
def ilikebuffers(x):
    n = len(x)
    buf = [np.zeros_like(x) for _ in range(n)]
    for i in prange(n):
        buf[i] = x + i
    return buf

a = np.ones((10,))

out = ilikebuffers(a)
[print(x) for x in out]

print("")

out = ilikebuffers.py_func(a)
[print(x) for x in out]

does this:

[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]
[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]
[4. 4. 4. 4. 4. 4. 4. 4. 4. 4.]
[4. 4. 4. 4. 4. 4. 4. 4. 4. 4.]
[6. 6. 6. 6. 6. 6. 6. 6. 6. 6.]
[6. 6. 6. 6. 6. 6. 6. 6. 6. 6.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]

[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]
[3. 3. 3. 3. 3. 3. 3. 3. 3. 3.]
[4. 4. 4. 4. 4. 4. 4. 4. 4. 4.]
[5. 5. 5. 5. 5. 5. 5. 5. 5. 5.]
[6. 6. 6. 6. 6. 6. 6. 6. 6. 6.]
[7. 7. 7. 7. 7. 7. 7. 7. 7. 7.]
[8. 8. 8. 8. 8. 8. 8. 8. 8. 8.]
[9. 9. 9. 9. 9. 9. 9. 9. 9. 9.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]

and it shouldn't. Suspect it's analysis of writing to the loop slots, but may be something else.

[stuartarchibald]

CC @DrTodd13

[DrTodd13]

@stuartarchibald I'm unable to replicate.

NUMBA_DEBUG_ARRAY_OPT=0 python /tmp/issue4294.py
[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]
[3. 3. 3. 3. 3. 3. 3. 3. 3. 3.]
[4. 4. 4. 4. 4. 4. 4. 4. 4. 4.]
[5. 5. 5. 5. 5. 5. 5. 5. 5. 5.]
[6. 6. 6. 6. 6. 6. 6. 6. 6. 6.]
[7. 7. 7. 7. 7. 7. 7. 7. 7. 7.]
[8. 8. 8. 8. 8. 8. 8. 8. 8. 8.]
[9. 9. 9. 9. 9. 9. 9. 9. 9. 9.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]

[1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]
[3. 3. 3. 3. 3. 3. 3. 3. 3. 3.]
[4. 4. 4. 4. 4. 4. 4. 4. 4. 4.]
[5. 5. 5. 5. 5. 5. 5. 5. 5. 5.]
[6. 6. 6. 6. 6. 6. 6. 6. 6. 6.]
[7. 7. 7. 7. 7. 7. 7. 7. 7. 7.]
[8. 8. 8. 8. 8. 8. 8. 8. 8. 8.]
[9. 9. 9. 9. 9. 9. 9. 9. 9. 9.]
[10. 10. 10. 10. 10. 10. 10. 10. 10. 10.]

[stuartarchibald]

@DrTodd13 thanks for trying, after poking about a bit more, I think this is related to core count. I'm going to guess that you ran this on a >10 core machine? Try this one, I get 4 distinct blocks of results, and ran this on a 4 core machine:

from numba import njit, prange
import numpy as np

@njit(parallel=True)
def ilikebuffers(x):
    n = 100
    buf = [np.zeros_like(x) for _ in range(n)]
    for i in prange(n):
        buf[i] = x + i
    return buf

a = np.ones((2,))

out = ilikebuffers(a)
[print(x) for x in out]

print("")

out = ilikebuffers.py_func(a)
[print(x) for x in out]

also if I set NUMBA_NUM_THREADS=8 I get 8 distinct blocks in the results.

Thanks.

[DrTodd13]

@stuartarchibald Confirmed. NUMBA_NUM_THREADS=1 gives me an output consisting of all 10's.