use of @jit and @njit makes function execute with false output

[La-Li-Lu-Le-Loa]

Reporting a bug

• [x ] I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• [x ] I have included below a minimal working reproducer (if you are unsure how
  to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

I searched an hour till i found out numbas `@jit `compilation is producing the error. (run the reproducable with and without numba decorators to see)

The sample data used are lists of arrays, which have the same size of the subarrays at the same index,
but a totally different size in the other indices. Therefore the inputs to func3() are converted to a numpy array of objects before passing them to func3().
Inside fun3() an empty numpy array (of exactly the same shape as the passed arrays) is created.
The array's subarray's elements will be "filled" with (or changed to) 0s or 1s depending on the first occouring condition.
This array will be returned.
My interpretation is, that at least the conditional part is completely ignored when @jit decorators are used.
I also included the "old" function (a list comprehension) which is a little slower but creates the correct output and a comparison funciton, to make things easier for us.

This could be a bug or false usage of the @jit decorator (though i reseached before opening this issue, but please excuse if the latter could be the case)

import numpy as np, numba as nb, time

# generate sample data
LEN = 10000; Amount_Of_Elements = 4320
temp = np.random.randint(Amount_Of_Elements*0.7,high=Amount_Of_Elements, size=LEN) 
RatiosUp         = np.array([np.random.uniform(size=rand) for rand in temp], dtype=object)
RatiosDown       = np.array([np.random.uniform(size=rand) for rand in temp], dtype=object)
UpPointsSlices   = np.array([np.random.uniform(size=rand) for rand in temp], dtype=object)
DownPointsSlices = np.array([np.random.uniform(size=rand) for rand in temp], dtype=object)  

@nb.njit 
def filter3(a,b):
   return a > b 

@nb.jit
def func3(RatiosUp, RatiosDown, UpPointsSlices, DownPointsSlices): 
   shapes = np.asarray([arr.size for arr in RatiosUp],dtype=object)
   result   = np.asarray([np.empty(s) for s in shapes],dtype=object)
   
   for i, shape in zip(range(len(result)), shapes): 
       for j in range(shape): 
           if filter3(RatiosUp[i][j],RatiosDown[i][j]):
               result[i][j] = 1
           elif filter3(RatiosDown[i][j],RatiosUp[i][j]):
               result[i][j] = 0
           elif filter3(UpPointsSlices[i][j],DownPointsSlices[i][j]): 
               result[i][j] = 0
           else:                       
               result[i][j] = 1 
   return result
# this is the original result without numba (== same result as above when used without decorators)
result0      = [ 
[1 if (ratUp >ratDown)  else 0 if (ratDown>ratUp) else  0 if (pointsDown>pointsUp) else 1    
              for ratUp,ratDown,pointsUp,pointsDown  
                          in zip(ratiosUpSlice,ratiosDownSlice,upPointsSlice,downPointsSlice)] 
                                        for ratiosUpSlice,ratiosDownSlice,upPointsSlice,downPointsSlice 
                                                   in zip(RatiosUp, RatiosDown, UpPointsSlices, DownPointsSlices)]
result1 = func3(RatiosUp, RatiosDown, UpPointsSlices, DownPointsSlices)

test = all(np.allclose(a,b, equal_nan=True) for a,b in zip(result0,result1.tolist()))
print(test)

[gmarkall]

Are you expecting this to print True for a correct answer? With the latest Numba master, I get:

$ python repro.py
<various warnings about looplifting and objectmode fallback>
True

If you get False, what version of Numba are you using? What is the output of numba -s?

[La-Li-Lu-Le-Loa]

If you get False, what version of Numba are you using? What is the output of numba -s?

I cant run numba -s on windows, but

import numba as nb
print(nb.__version__)

gives me
0.51.2
this seems to be the current version (because after updating its 0.51.2 again) or am I wrong?

reinstalled numba and still get a false :(

[gmarkall]

I cant run numba -s on windows, but

numba -s should work fine on Windows as long as it is on your path - what happens when you try to run it?

import numba as nb
print(nb.__version__)

gives me
0.51.2
this seems to be the current version (because after updating its 0.51.2 again) or am I wrong?

That's the current version, you're correct.

reinstalled numba and still get a false :(

I have tried in a fresh conda environment on Windows:

conda create -n numba512 python=3.8 numba=0.51.2
conda activate numba512
numba -s
# ... Produces a summary of the system
python issue-6247.py
# prints "True"

How are you installing Numba? If you create a new virtualenv or conda env and install Numba into that, do you get a True?

[La-Li-Lu-Le-Loa]

I dont use conda, since it always slows down my whole computer. (gave it a few tries), instead use spyder (created a shortcut).

There seems to be a bug, so i reinstalled python completely and got the same(python 3.7) .
I tried it in python 3.8, but now I get:

    super(CastInstr, self).__init__(parent, typ, op, [val], name=name)

TypeError: Failed in object mode pipeline (step: object mode frontend)
Failed in object mode pipeline (step: object mode backend)
super(type, obj): obj must be an instance or subtype of type

in the line in which func3() is called

[gmarkall]

It looks like there's still something mixed up about your Python / Numba installation.

How did you reinstall Python? How are you using Spyder? (I don't use Spyder, but my limited understanding is that it's an IDE, rather than a package manager, so I'm not sure how Spyder is something you can use instead of conda).

[La-Li-Lu-Le-Loa]

It looks like there's still something mixed up about your Python / Numba installation.

How did you reinstall Python? How are you using Spyder? (I don't use Spyder, but my limited understanding is that it's an IDE, rather than a package manager, so I'm not sure how Spyder is something you can use instead of conda).

I just did pip install spyder and then created a desktop shortcut to the spyder3.exe file in the python37 folder.
when I click on it, it opens Spyder IDE.

I reinstalled python by deleting executing a normal windows deinstallation on python and the python laucher.
and afterwards i deleted the python37 folder.
=> did I forgot something?

[gmarkall]

If you do:

import numba
from numba.misc import numba_sysinfo
numba_sysinfo.display_sysinfo()

What is the output?

[La-Li-Lu-Le-Loa]

If you do:

import numba
from numba.misc import numba_sysinfo
numba_sysinfo.display_sysinfo()

What is the output?

--------------------------
__Time Stamp__
Report started (local time)                   : 2020-09-19 00:16:35.961582
UTC start time                                : 2020-09-18 22:16:35.961582
Running time (s)                              : 3.511599

__Hardware Information__
Machine                                       : AMD64
CPU Name                                      : sandybridge
CPU Count                                     : 4
Number of accessible CPUs                     : 4
List of accessible CPUs cores                 : 0 1 2 3
CFS Restrictions (CPUs worth of runtime)      : None

CPU Features                                  : 64bit aes avx cmov cx16 cx8 fxsr
                                                mmx pclmul popcnt sahf sse sse2
                                                sse3 sse4.1 sse4.2 ssse3 xsave
                                                xsaveopt

Memory Total (MB)                             : 16052
Memory Available (MB)                         : 11669

__OS Information__
Platform Name                                 : Windows-10-10.0.18362-SP0
Platform Release                              : 10
OS Name                                       : Windows
OS Version                                    : 10.0.18362
OS Specific Version                           : 10 10.0.18362 SP0 Multiprocessor Free
Libc Version                                  : ?

__Python Information__
Python Compiler                               : MSC v.1927 64 bit (AMD64)
Python Implementation                         : CPython
Python Version                                : 3.8.6rc1
Python Locale                                 : de_DE.cp1252

__LLVM Information__
LLVM Version                                  : 10.0.1

__CUDA Information__
CUDA Device Initialized                       : False
CUDA Driver Version                           : ?
CUDA Detect Output:
None
CUDA Librairies Test Output:
None

__ROC information__
ROC Available                                 : False
ROC Toolchains                                : None
HSA Agents Count                              : 0
HSA Agents:
None
HSA Discrete GPUs Count                       : 0
HSA Discrete GPUs                             : None

__SVML Information__
SVML State, config.USING_SVML                 : False
SVML Library Loaded                           : False
llvmlite Using SVML Patched LLVM              : True
SVML Operational                              : False

__Threading Layer Information__
TBB Threading Layer Available                 : False
+--> Disabled due to Unknown import problem.
OpenMP Threading Layer Available              : True
+-->Vendor: MS
Workqueue Threading Layer Available           : True
+-->Workqueue imported successfully.

__Numba Environment Variable Information__
None found.

__Conda Information__
Conda not available.

__Installed Packages__
Package                       Version
----------------------------- ---------
alabaster                     0.7.12
astroid                       2.4.2
async-generator               1.10
atomicwrites                  1.4.0
attrs                         20.2.0
autopep8                      1.5.4
Babel                         2.8.0
backcall                      0.2.0
bcrypt                        3.2.0
bleach                        3.2.1
certifi                       2020.6.20
cffi                          1.14.2
chardet                       3.0.4
cloudpickle                   1.6.0
colorama                      0.4.3
cryptography                  3.1
decorator                     4.4.2
defusedxml                    0.6.0
diff-match-patch              20200713
docutils                      0.16
entrypoints                   0.3
flake8                        3.8.3
helpdev                       0.7.1
idna                          2.10
imagesize                     1.2.0
intervaltree                  3.1.0
ipykernel                     5.3.4
ipython                       7.18.1
ipython-genutils              0.2.0
isort                         5.5.2
jedi                          0.17.1
Jinja2                        2.11.2
joblib                        0.16.0
jsonschema                    3.2.0
jupyter-client                6.1.7
jupyter-core                  4.6.3
jupyterlab-pygments           0.1.1
keyring                       21.4.0
lazy-object-proxy             1.4.3
llvmlite                      0.34.0
MarkupSafe                    1.1.1
mccabe                        0.6.1
mistune                       0.8.4
nbclient                      0.5.0
nbconvert                     6.0.3
nbformat                      5.0.7
nest-asyncio                  1.4.0
numba                         0.51.2
numpy                         1.19.2
numpydoc                      1.1.0
packaging                     20.4
pandas                        1.1.2
pandocfilters                 1.4.2
paramiko                      2.7.2
parso                         0.7.0
pathtools                     0.1.2
pexpect                       4.8.0
pickleshare                   0.7.5
pip                           20.2.1
pluggy                        0.13.1
prompt-toolkit                3.0.7
psutil                        5.7.2
ptyprocess                    0.6.0
pycodestyle                   2.6.0
pycparser                     2.20
pydocstyle                    5.1.1
pyflakes                      2.2.0
Pygments                      2.7.1
pylint                        2.6.0
PyNaCl                        1.4.0
pyparsing                     2.4.7
PyQt5                         5.12.3
PyQt5-sip                     12.8.1
PyQtWebEngine                 5.12.1
pyrsistent                    0.17.3
python-dateutil               2.8.1
python-jsonrpc-server         0.4.0
python-language-server        0.35.1
pytz                          2020.1
pywin32                       228
pywin32-ctypes                0.2.0
pyzmq                         19.0.2
QDarkStyle                    2.8.1
QtAwesome                     0.7.2
qtconsole                     4.7.7
QtPy                          1.9.0
requests                      2.24.0
rope                          0.17.0
scikit-learn                  0.23.2
scipy                         1.5.2
setuptools                    49.2.1
six                           1.15.0
sklearn                       0.0
snowballstemmer               2.0.0
sortedcontainers              2.2.2
Sphinx                        3.2.1
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        1.0.3
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.4
spyder                        4.1.5
spyder-kernels                1.9.4
testpath                      0.4.4
threadpoolctl                 2.1.0
toml                          0.10.1
tornado                       6.0.4
traitlets                     5.0.4
ujson                         3.2.0
urllib3                       1.25.10
watchdog                      0.10.3
wcwidth                       0.2.5
webencodings                  0.5.1
wrapt                         1.12.1
yapf                          0.30.0

No errors reported.


__Warning log__
Warning (cuda): CUDA driver library cannot be found or no CUDA enabled devices are present.
Exception class: <class 'numba.cuda.cudadrv.error.CudaSupportError'>
Warning (roc): Error initialising ROC: No ROC toolchains found.
Warning (roc): No HSA Agents found, encountered exception when searching: Error at driver init: 

HSA is not currently supported on this platform (win32).
:
Warning: Conda not available.
 Error was [WinError 2] Das System kann die angegebene Datei nicht finden

--------------------------------------------------------------------------------
If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.
=============================================================


WARNING: You are using pip version 20.2.1; however, version 20.2.3 is available.
You should consider upgrading via the 'c:\users\user\appdata\local\programs\python\python38\python.exe -m pip install --upgrade pip' command.

[La-Li-Lu-Le-Loa]

reinstalled python again, this time i looked for any single folder in the appdata folder. now it works on 3.8

[esc]

@La-Li-Lu-Le-Loa thanks for reporting back, happy it works for you now.