Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check if variable is None inside njitted function is neither True nor False with parallel=True #5747

Open
2 tasks done
jtilly opened this issue May 25, 2020 · 5 comments
Open
2 tasks done

Check if variable is None inside njitted function is neither True nor False with parallel=True #5747

jtilly opened this issue May 25, 2020 · 5 comments
Labels
bug - incorrect behavior Bugs: incorrect behavior ParallelAccelerator SSA Problem due to SSA (or lack of)

Comments

@jtilly
Copy link
Contributor

jtilly commented May 25, 2020 •
edited

This problem is new in v0.49.x. It only occurs when using parallel=True.

I have two functions test and test2 that are identical except for that test checks if weight is not None and test2 checks if weight is None.

test produces the correct result, i.e. it sums up the weights (=2.5). test2 thinks that weight is not None is neither True nor False and returns 0.

import numpy as np
from numba import njit, prange

@njit(parallel=True)
def test(y, weights):
    sum_weights = 0.0
    if weights is not None:
        for i in prange(len(y)):
            sum_weights += weights[i]
    else:
        for i in prange(len(y)):
            sum_weights += 1.0

    return sum_weights

@njit(parallel=True)  # `parallel=False` works!
def test2(y, weights):
    sum_weights = 0.0
    if weights is None:  # `if False:` works!
        for i in prange(len(y)):
            sum_weights += 1.0
    else:
    	for i in prange(len(y)):
            sum_weights += weights[i]

    return sum_weights

y = np.array([0.4] * 5)
weights = np.array([0.5] * 5)

print(test(y, weights))  # 2.5
print(test2(y, weights))  # 0.0

Environments

I'm running this on Linux with the conda-forge versions of numba and numpy.

conda create -y -c conda-forge -n numba0490 numba=0.49.0
conda create -y -c conda-forge -n numba0491 numba=0.49.1
conda create -y -c conda-forge -n numba0480 numba=0.48.0
@esc
Copy link
Member

esc commented May 25, 2020

@jtilly thanks for submitting this to the Numba issue tracker. I will label it as needing triage for now.

@esc
Copy link
Member

esc commented May 25, 2020 •
edited

O.K. it seems like I can reproduce this locally, it is likely a bug in the parallel accelerator. Probably one of the next steps will be to use git bisect to figure out when the regression was introduced to the Numba source tree.

@jtilly
Copy link
Contributor Author

jtilly commented May 25, 2020

O.K. it seems like I can reproduce this locally, it is likely a bug in the parallel accelerator. Probably one of the next steps will be to use git bisect to figure out when the regression was introduced to the Numba source tree.

I took a stab at it. Last good commit that I could find: 868b8e3

First commit that throws no error but produces the result I'm reporting above: 2d33fb1

2d33fb11617292052017819e53c6b6120b540971 (HEAD, refs/bisect/bad) Fix most of generators
4de2cf3c2e8ddca7ee68301e5d07a55aed55ecee (refs/bisect/skip-4de2cf3c2e8ddca7ee68301e5d07a55aed55ecee) Cleanup
7a881abb5c428d6c0be7be73a89e8c2444c211bb (refs/bisect/skip-7a881abb5c428d6c0be7be73a89e8c2444c211bb) Fix missing symbol problem because we changed the function id
15289c014aae6e15f7338bc4bbae333ca5fb5eed (refs/bisect/skip-15289c014aae6e15f7338bc4bbae333ca5fb5eed) Cleanup
571828d1f976435bf077b9fd89d03d3e9673069d (refs/bisect/skip-571828d1f976435bf077b9fd89d03d3e9673069d) Fix SSA complaining about undefined variable
909c4e34f4a4df1aadb727f071a46f99ac83dd2a (refs/bisect/skip-909c4e34f4a4df1aadb727f071a46f99ac83dd2a) Cannot use assignlist because the IR is being mutated
00c0706043f0f57b5a1f95a133910bb5648946e0 (refs/bisect/skip-00c0706043f0f57b5a1f95a133910bb5648946e0) Fix forgotten handling for assign var to var
5abe69688b75e2853630269a96d460147bd26461 (refs/bisect/skip-5abe69688b75e2853630269a96d460147bd26461) Add test for problems in phi propagation i.e. not using iterated domfronts
01477ddda98994285cc815406d917539afe04169 (refs/bisect/skip-01477ddda98994285cc815406d917539afe04169) Fix searching for phi node locally
1f65e1a404abaa7fdf287f6818e7d342572d728f (refs/bisect/skip-1f65e1a404abaa7fdf287f6818e7d342572d728f) Use iterated domfronts
759a2cf4540196018997b6919efe9beabb8fe64b (refs/bisect/skip-759a2cf4540196018997b6919efe9beabb8fe64b) Fix SSA falling into infinite loop because it can't find definition
9db91ede5b22c682bc7572726ceb2f33f90acf3f (refs/bisect/skip-9db91ede5b22c682bc7572726ceb2f33f90acf3f) Add SSA tests
e568c62ae7d094365065856dfb17002b73a456cc (refs/bisect/skip-e568c62ae7d094365065856dfb17002b73a456cc) Fix uninitialized variable when used with SSA
7c14930a5776e149925264201e4b730fab4cd205 (refs/bisect/skip-7c14930a5776e149925264201e4b730fab4cd205) Fix problem with first assignment being renamed away
085031f77884dd8122c6e79d2fd4c4d2ff7e6e6f (refs/bisect/skip-085031f77884dd8122c6e79d2fd4c4d2ff7e6e6f) Make ReconstructSSA pass
536bb3d531be849f35519430860698ef914915df (refs/bisect/skip-536bb3d531be849f35519430860698ef914915df) Add phis stripping pass before IRLegalisation
bf06d0d7f0fb76e54b0b869b70a3dfcfa6582641 (refs/bisect/skip-bf06d0d7f0fb76e54b0b869b70a3dfcfa6582641) Fix problems with Arguments and local block assignments selection
706f6b1098bcef48cea068752bce094676fc42c4 (refs/bisect/skip-706f6b1098bcef48cea068752bce094676fc42c4) Adapt typeinfer and lowering for SSA
76b606ddfde801b8ea833120fc6c3d6691bda0e6 (refs/bisect/skip-76b606ddfde801b8ea833120fc6c3d6691bda0e6) Implement DF based SSA reconstruction
868b8e3e8d034dac0440b75ca31595e07f632d27 (refs/bisect/good-868b8e3e8d034dac0440b75ca31595e07f632d27) Merge pull request #5330 from sklam/enh/refactor_parfor_lowering

The diff between good and bad is here.

For commits in between I get two types of error messages:

Finished processing dependencies for numba==0.49.0.dev0+717.g571828d1f
Traceback (most recent call last):
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/errors.py", line 720, in new_error_context
    yield
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 267, in lower_block
    self.lower_inst(inst)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 339, in lower_inst
    val = self.lower_assign(ty, inst)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 513, in lower_assign
    return self.lower_expr(ty, value)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 962, in lower_expr
    return self.lower_binop(resty, expr, expr.immutable_fn)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 583, in lower_binop
    lhs = self.loadvar(lhs.name)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 1197, in loadvar
    ptr = self.getvar(name)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 1191, in getvar
    return self.varmap[name]
KeyError: 'sum_weights.3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/test.py", line 31, in <module>
    print(test(y, w))  # 2.5
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 420, in _compile_for_args
    raise e
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 353, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 768, in compile
    cres = self._compiler.compile(args, return_type)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 77, in compile
    status, retval = self._compile_cached(args, return_type)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 91, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 104, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 555, in compile_extra
    return pipeline.compile_extra(func)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 331, in compile_extra
    return self._compile_bytecode()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 393, in _compile_bytecode
    return self._compile_core()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 373, in _compile_core
    raise e
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 364, in _compile_core
    pm.run(self.state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 341, in run
    raise patched_exception
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 332, in run
    self._runPass(idx, pass_inst, state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 291, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 264, in check
    mangled = func(compiler_state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/typed_passes.py", line 421, in run_pass
    NativeLowering().run_pass(state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/typed_passes.py", line 363, in run_pass
    lower.lower()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 180, in lower
    self.lower_normal_function(self.fndesc)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 227, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 252, in lower_function_body
    self.lower_block(block)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 267, in lower_block
    self.lower_inst(inst)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 455, in lower_inst
    func(self, inst)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/parfors/parfor_lowering.py", line 231, in _lower_parfor_parallel
    func, func_args, func_sig, redargstartdim, func_arg_types = _create_gufunc_for_parfor_body(
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/parfors/parfor_lowering.py", line 1160, in _create_gufunc_for_parfor_body
    kernel_func = compiler.compile_ir(
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 618, in compile_ir
    return pipeline.compile_ir(func_ir=func_ir, lifted=lifted,
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 341, in compile_ir
    return self._compile_ir()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 400, in _compile_ir
    return self._compile_core()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 373, in _compile_core
    raise e
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 364, in _compile_core
    pm.run(self.state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 341, in run
    raise patched_exception
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 332, in run
    self._runPass(idx, pass_inst, state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 291, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 264, in check
    mangled = func(compiler_state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/typed_passes.py", line 421, in run_pass
    NativeLowering().run_pass(state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/typed_passes.py", line 363, in run_pass
    lower.lower()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 180, in lower
    self.lower_normal_function(self.fndesc)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 227, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 252, in lower_function_body
    self.lower_block(block)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/lowering.py", line 267, in lower_block
    self.lower_inst(inst)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/errors.py", line 727, in new_error_context
    reraise(type(newerr), newerr, tb)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+717.g571828d1f-py3.8-linux-x86_64.egg/numba/core/utils.py", line 78, in reraise
    raise value
numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: nopython mode backend)
Failed in nopython mode pipeline (step: nopython mode backend)
'sum_weights.3'

File "../../../tmp/test.py", line 9:
def test(y, weights):
    <source elided>
        for i in prange(len(y)):
            sum_weights += weights[i]
            ^

[1] During: lowering "$sum_weights.1.90 = inplace_binop(fn=<built-in function iadd>, immutable_fn=<built-in function add>, lhs=sum_weights.3, rhs=$36binary_subscr.5, static_lhs=Undefined, static_rhs=Undefined)" at /tmp/test.py (9)
[2] During: lowering "id=1[LoopNest(index_variable = parfor_index.6, range = (0, y_size0.1, 1))]{28: <ir.Block at /tmp/test.py (8)>}Var(parfor_index.6, test.py:8)" at /tmp/test.py (8)

Finished processing dependencies for numba==0.49.0.dev0+703.g76b606ddf
-----------------------------------BEFORE SSA-----------------------------------
label 0:
    y = arg(0, name=y)                       ['y']
    weights = arg(1, name=weights)           ['weights']
    $const2.0 = const(float, 0.0)            ['$const2.0']
    sum_weights = $const2.0                  ['$const2.0', 'sum_weights']
    $const8.2 = const(NoneType, None)        ['$const8.2']
    $10compare_op.3 = weights is not $const8.2 ['$10compare_op.3', '$const8.2', 'weights']
    branch $10compare_op.3, 14, 46           ['$10compare_op.3']
label 14:
    $14load_global.0 = global(prange: <class 'numba.misc.special.prange'>) ['$14load_global.0']
    $16load_global.1 = global(len: <built-in function len>) ['$16load_global.1']
    $20call_function.3 = call $16load_global.1(y, func=$16load_global.1, args=[Var(y, test.py:6)], kws=(), vararg=None) ['$16load_global.1', '$20call_function.3', 'y']
    $22call_function.4 = call $14load_global.0($20call_function.3, func=$14load_global.0, args=[Var($20call_function.3, test.py:8)], kws=(), vararg=None) ['$14load_global.0', '$20call_function.3', '$22call_function.4']
    $24get_iter.5 = getiter(value=$22call_function.4) ['$22call_function.4', '$24get_iter.5']
    $phi26.0 = $24get_iter.5                 ['$24get_iter.5', '$phi26.0']
    jump 26                                  []
label 26:
    $26for_iter.1 = iternext(value=$phi26.0) ['$26for_iter.1', '$phi26.0']
    $26for_iter.2 = pair_first(value=$26for_iter.1) ['$26for_iter.1', '$26for_iter.2']
    $26for_iter.3 = pair_second(value=$26for_iter.1) ['$26for_iter.1', '$26for_iter.3']
    $phi28.1 = $26for_iter.2                 ['$26for_iter.2', '$phi28.1']
    branch $26for_iter.3, 28, 44             ['$26for_iter.3']
label 28:
    i = $phi28.1                             ['$phi28.1', 'i']
    $36binary_subscr.5 = getitem(value=weights, index=i) ['$36binary_subscr.5', 'i', 'weights']
    $38inplace_add.6 = inplace_binop(fn=<built-in function iadd>, immutable_fn=<built-in function add>, lhs=sum_weights, rhs=$36binary_subscr.5, static_lhs=Undefined, static_rhs=Undefined) ['$36binary_subscr.5', '$38inplace_add.6', 'sum_weights']
    sum_weights = $38inplace_add.6           ['$38inplace_add.6', 'sum_weights']
    jump 26                                  []
label 44:
    jump 72                                  []
label 46:
    $46load_global.0 = global(prange: <class 'numba.misc.special.prange'>) ['$46load_global.0']
    $48load_global.1 = global(len: <built-in function len>) ['$48load_global.1']
    $52call_function.3 = call $48load_global.1(y, func=$48load_global.1, args=[Var(y, test.py:6)], kws=(), vararg=None) ['$48load_global.1', '$52call_function.3', 'y']
    $54call_function.4 = call $46load_global.0($52call_function.3, func=$46load_global.0, args=[Var($52call_function.3, test.py:11)], kws=(), vararg=None) ['$46load_global.0', '$52call_function.3', '$54call_function.4']
    $56get_iter.5 = getiter(value=$54call_function.4) ['$54call_function.4', '$56get_iter.5']
    $phi58.0 = $56get_iter.5                 ['$56get_iter.5', '$phi58.0']
    jump 58                                  []
label 58:
    $58for_iter.1 = iternext(value=$phi58.0) ['$58for_iter.1', '$phi58.0']
    $58for_iter.2 = pair_first(value=$58for_iter.1) ['$58for_iter.1', '$58for_iter.2']
    $58for_iter.3 = pair_second(value=$58for_iter.1) ['$58for_iter.1', '$58for_iter.3']
    $phi60.1 = $58for_iter.2                 ['$58for_iter.2', '$phi60.1']
    branch $58for_iter.3, 60, 72             ['$58for_iter.3']
label 60:
    i = $phi60.1                             ['$phi60.1', 'i']
    $const64.3 = const(float, 1.0)           ['$const64.3']
    $66inplace_add.4 = inplace_binop(fn=<built-in function iadd>, immutable_fn=<built-in function add>, lhs=sum_weights, rhs=$const64.3, static_lhs=Undefined, static_rhs=Undefined) ['$66inplace_add.4', '$const64.3', 'sum_weights']
    sum_weights = $66inplace_add.4           ['$66inplace_add.4', 'sum_weights']
    jump 58                                  []
label 72:
    $74return_value.1 = cast(value=sum_weights) ['$74return_value.1', 'sum_weights']
    return $74return_value.1                 ['$74return_value.1']

None
================================================================================
Traceback (most recent call last):
  File "/tmp/test.py", line 31, in <module>
    print(test(y, w))  # 2.5
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 420, in _compile_for_args
    raise e
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 353, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 768, in compile
    cres = self._compiler.compile(args, return_type)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 77, in compile
    status, retval = self._compile_cached(args, return_type)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 91, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/dispatcher.py", line 104, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 549, in compile_extra
    return pipeline.compile_extra(func)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 329, in compile_extra
    return self._compile_bytecode()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 391, in _compile_bytecode
    return self._compile_core()
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 371, in _compile_core
    raise e
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler.py", line 362, in _compile_core
    pm.run(self.state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 341, in run
    raise patched_exception
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 332, in run
    self._runPass(idx, pass_inst, state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 291, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/compiler_machinery.py", line 264, in check
    mangled = func(compiler_state)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/untyped_passes.py", line 85, in run_pass
    func_ir = interp.interpret(bc)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/interpreter.py", line 142, in interpret
    return recontruct_ssa(fir)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/ssa.py", line 21, in recontruct_ssa
    newblocks = _run_ssa(fir.blocks)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/ssa.py", line 45, in _run_ssa
    blocks = _fix_ssa_vars(blocks, varname, defmap)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/ssa.py", line 59, in _fix_ssa_vars
    newblocks = _run_block_rewrite(blocks, states, _FixSSAVars())
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/ssa.py", line 121, in _run_block_rewrite
    for stmt in _run_sbaa_block_pass(states, blk, handler):
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/ssa.py", line 141, in _run_sbaa_block_pass
    ret = handler.on_assign(states, stmt)
  File "/opt/local/conda/envs/numba-dev/lib/python3.8/site-packages/numba-0.49.0.dev0+703.g76b606ddf-py3.8-linux-x86_64.egg/numba/core/ssa.py", line 213, in on_assign
    replmap = {states['varname']: phidef.target}
AttributeError: Failed in nopython mode pipeline (step: analyzing bytecode)
'Var' object has no attribute 'target'

@esc
Copy link
Member

esc commented May 26, 2020

@jtilly thanks for doing the bisect, looking at the output, it appears like there may be several intertwined issues at play here. The commit 2d33fb1 seems to have been part of #5351 -- so this would be evidence that this might be fallout from the recent introduction of the SSA algorithm.

@esc esc added the SSA Problem due to SSA (or lack of) label May 26, 2020
@stuartarchibald stuartarchibald added bug - incorrect behavior Bugs: incorrect behavior and removed bug labels Sep 11, 2020
@stuartarchibald
Copy link
Contributor

CC @DrTodd13, I can reproduce this on mainline for ~0.54 series releases. Doesn't appear to be related to the branch pruner, could be an SSA interaction with parfors, perhaps a versioned variable that needs to be unversioned?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug - incorrect behavior Bugs: incorrect behavior ParallelAccelerator SSA Problem due to SSA (or lack of)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@esc @stuartarchibald @jtilly