Cannot lower setitem on a record in a parallel loop

[GuillaumeLeclerc]

Reporting a bug

When running this code:

import numba
import numpy as np

print(numba.__version__)
# => 0.52.0

state_dtype = np.dtype([
    ('var', np.int32)
 ])

@numba.njit(inline=True)
def inner_function(state):
    state['var'] = 1

@numba.njit(parallel=True)
def outer_function(states):
    for i in numba.prange(1):
        inner_function(states[i])

outer_function(np.ones(shape=1, dtype=state_dtype))

numba yields
No definition for lowering <built-in function setitem>(Record(var[type=int32;offset=0];4;False), Literal[str](var), Literal[int](1)) -> none, which seems to be the correct signature and I'm not so sure why there is no lowering available for that situation.

When disabling parallel or inlining the code compiles and produces correct results.

• I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

[gmarkall]

Many thanks for the issue report - I can reproduce this, with a minor change to the reproducer:

import numba
import numpy as np

print(numba.__version__)
# => 0.52.0

state_dtype = np.dtype([
    ('var', np.int32)
 ])

@numba.njit(inline='always')
def inner_function(state):
    state['var'] = 1

@numba.njit(parallel=True)
def outer_function(states):
    for i in numba.prange(1):
        inner_function(states[i])

outer_function(np.ones(shape=1, dtype=state_dtype))

This is because setting inline=True resulted in the error:

ValueError: Failed in nopython mode pipeline (step: inline inlinable functions)
kwarg 'inline' must be one of the strings 'always' or 'never', or it can be a callable that returns True/False. Found value True

so I changed it to inline=always`.

As a workaround, you can spell the setitem as a setattr, at least for this reproducer:

import numba
import numpy as np

print(numba.__version__)
# => 0.52.0

state_dtype = np.dtype([
    ('var', np.int32)
 ])

@numba.njit(inline='always')
def inner_function(state):
    state.var = 2

@numba.njit(parallel=True)
def outer_function(states):
    for i in numba.prange(1):
        inner_function(states[i])

data = np.ones(shape=1, dtype=state_dtype)

outer_function(data)

print(data)

I edited the setting of data from 1 to 2 so the effect of the inlined function would be visible. This outputs for me:

$ python war.py 
0.53.0dev0+739.gf6e735f71
OMP: Info #274: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
[(2,)]

[stuartarchibald]

I don't think this has anything to do with the inliner.

from numba import njit, prange
import numpy as np

state_dtype = np.dtype([
    ('var', np.int32)
 ])

def outer_function(states):
    for i in prange(1):
        states[i]['var'] = 1

print("interpreter", outer_function(np.ones(shape=1, dtype=state_dtype)))
print("njit", njit(outer_function)(np.ones(shape=1, dtype=state_dtype)))
print("parallel accelerator", njit(parallel=True)(outer_function)(np.ones(shape=1, dtype=state_dtype)))

it's just the last line that fails, same way as above.

[stuartarchibald]

CC @DrTodd13 ^ thanks.

[GuillaumeLeclerc]

That's weird because on my machine having inline='never' compiles and run fine

[stuartarchibald]

That's weird because on my machine having inline='never' compiles and run fine

The parallel transforms can't "look" through a function call, so if the function remains as a call by virtue of it not being inlined the problem would never be hit. Essentially the function would be compiled by "normal" jit (which demonstrably works fine) and the parallel region would just call that function.

As an aside, what's the reasoning behing inlining this function at the Numba IR level? If it's performance, LLVM would probably inline it anyway, there's some notes on what Numba inlining does and its uses here: https://numba.readthedocs.io/en/stable/developer/inlining.html#notes-on-inlining .

[GuillaumeLeclerc]

Thanks for the explanation @stuartarchibald it makes sense now!

Yeah it is performance. In some instances I've seen up to 30% difference. However I only benchmarked without the parallel flag.

[DrTodd13]

@stuartarchibald I guess StaticSetItem is supported for record array writes and not SetItem?