Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llvmlite, numba errors #7

Closed
Screenium opened this issue Mar 4, 2018 · 4 comments
Closed

llvmlite, numba errors #7

Screenium opened this issue Mar 4, 2018 · 4 comments

Comments

@Screenium
Copy link 

Traceback (most recent call last):
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\errors.py", line 259, in new_error_context
    yield
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 216, in lower_block
    self.lower_inst(inst)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 265, in lower_inst
    val = self.lower_assign(ty, inst)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 411, in lower_assign
    return self.lower_expr(ty, value)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 743, in lower_expr
    res = self.lower_call(resty, expr)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 709, in lower_call
    res = impl(self.builder, argvals)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\targets\base.py", line 1043, in __call__
    return self._imp(self._context, builder, self._sig, args)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\targets\arrayobj.py", line 3158, in numpy_zeros_nd
    ary = _empty_nd_impl(context, builder, arrtype, shapes)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\targets\arrayobj.py", line 3045, in _empty_nd_impl
    arrlen = builder.mul(arrlen, s)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\llvmlite\ir\builder.py", line 24, in wrapped
    % (lhs.type, rhs.type))
ValueError: Operands must be the same type, got (i64, i32)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/Predator/Documents/WORK/Project_CT/radio/radioInput.py", line 48, in <module>
    (lunaset.train >> crop_pipeline).run()
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\pipeline.py", line 1088, in run
    for _ in self.gen_batch(*args, **kwargs):
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\pipeline.py", line 1035, in gen_batch
    batch_res = self._exec(batch)
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\pipeline.py", line 579, in _exec
    batch_res = self._exec_all_actions(batch)
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\pipeline.py", line 565, in _exec_all_actions
    batch = self._exec_one_action(batch, _action, _action_args, _action['kwargs'])
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\pipeline.py", line 516, in _exec_one_action
    batch = action_method(*args, **kwargs)
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\decorators.py", line 37, in _action_wrapper
    _res = action_method(action_self, *args, **kwargs)
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\preprocessing\ct_masked_batch.py", line 719, in sample_dump
    nodules = self.sample_nodules(batch_size=batch_size, nodule_size=nodule_size, share=share, **kwargs)
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\dataset\dataset\decorators.py", line 37, in _action_wrapper
    _res = action_method(action_self, *args, **kwargs)
  File "C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\preprocessing\ct_masked_batch.py", line 650, in sample_nodules
    images = get_nodules_numba(self.images, nodules_st_pos, nodule_size)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\dispatcher.py", line 307, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\dispatcher.py", line 579, in compile
    cres = self._compiler.compile(args, return_type)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\dispatcher.py", line 80, in compile
    flags=flags, locals=self.locals)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 779, in compile_extra
    return pipeline.compile_extra(func)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 362, in compile_extra
    return self._compile_bytecode()
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 738, in _compile_bytecode
    return self._compile_core()
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 725, in _compile_core
    res = pm.run(self.status)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 248, in run
    raise patched_exception
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 240, in run
    stage()
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 658, in stage_nopython_backend
    self._backend(lowerfn, objectmode=False)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 613, in _backend
    lowered = lowerfn()
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 600, in backend_nopython_mode
    self.flags)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\compiler.py", line 898, in native_lowering_stage
    lower.lower()
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 135, in lower
    self.lower_normal_function(self.fndesc)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 176, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 201, in lower_function_body
    self.lower_block(block)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 216, in lower_block
    self.lower_inst(inst)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\errors.py", line 265, in new_error_context
    six.reraise(type(newerr), newerr, sys.exc_info()[2])
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\errors.py", line 259, in new_error_context
    yield
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 216, in lower_block
    self.lower_inst(inst)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 265, in lower_inst
    val = self.lower_assign(ty, inst)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 411, in lower_assign
    return self.lower_expr(ty, value)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 743, in lower_expr
    res = self.lower_call(resty, expr)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\lowering.py", line 709, in lower_call
    res = impl(self.builder, argvals)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\targets\base.py", line 1043, in __call__
    return self._imp(self._context, builder, self._sig, args)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\targets\arrayobj.py", line 3158, in numpy_zeros_nd
    ary = _empty_nd_impl(context, builder, arrtype, shapes)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\numba\targets\arrayobj.py", line 3045, in _empty_nd_impl
    arrlen = builder.mul(arrlen, s)
  File "C:\Users\Predator\AppData\Local\Programs\Python\Python35\lib\site-packages\llvmlite\ir\builder.py", line 24, in wrapped
    % (lhs.type, rhs.type))
numba.errors.LoweringError: Failed at nopython (nopython mode backend)
Operands must be the same type, got (i64, i32)
File "radio\preprocessing\ct_masked_batch.py", line 59
[1] During: lowering "$0.20 = call $0.2($0.19, func=$0.2, args=[Var($0.19, C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\preprocessing\ct_masked_batch.py (59))], vararg=None, kws=[])" at C:\Users\Predator\Documents\WORK\Project_CT\radio\radio\preprocessing\ct_masked_batch.py (59)

The code above was tested on a very small dataset of pictures from LUNA16 dataset after 'lunaset.train >> crop_pipeline).run()' just freezes my PC for eternity testing on the whole dataset. Restarts even (after long time of being freezed). Running on win10 x64 with i7 processor.

Everything is up to date. Upgraded to newest numpy+mkl,numba,llvmlite. Running the python 3.5.

Any one experiencing the same problem?
@Screenium
Copy link
Author

The code is from the tutorial 4

@AlexeyKozhevin
Copy link

AlexeyKozhevin commented Mar 5, 2018 •
edited
Loading

@Screenium
The error arises because of different default types of integer numbers in different operating systems. I guess you use Windows so faced that problem. We've already fixed it and will add in the following release. Now you can change function get_nodules_numba in radio/preprocessing/ct_masked_batch.py in the following way:

@njit(nogil=True)
def get_nodules_numba(data, positions, size):
    size = size.astype(np.int64)
    out_arr = np.zeros((positions.shape[0], size[0], size[1], size[2]))
    n_positions = positions.shape[0]
    for i in range(n_positions):
        out_arr[i, :, :, :] = data[positions[i, 0]: positions[i, 0] + size[0],
                                   positions[i, 1]: positions[i, 1] + size[1],
                                   positions[i, 2]: positions[i, 2] + size[2]]

    return out_arr.reshape(n_positions * size[0], size[1], size[2])

Best,
Alexey

@Screenium
Copy link
Author

Thank you for answer. It did indeed help to move from the issue how ever my windows still freezes to death while running the pipeline. Switched to mac but following error has yielded :

Traceback (most recent call last):
  File "/Users/Welcome/Documents/work/radio/radIO.py", line 82, in <module>
    batch = combine_pipeline.next_batch()
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/pipeline.py", line 1057, in next_batch
    batch_res = self.next_batch(*self._lazy_run[0], **self._lazy_run[1])
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/pipeline.py", line 1062, in next_batch
    batch_res = next(self._batch_generator)
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/pipeline.py", line 1033, in gen_batch
    for batch in batch_generator:
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/base.py", line 162, in gen_batch
    for ix_batch in self.index.gen_batch(batch_size, shuffle, n_epochs, drop_last):
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/dsindex.py", line 276, in gen_batch
    batch = self.next_batch(batch_size, shuffle, n_epochs, drop_last, iter_params)
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/dsindex.py", line 250, in next_batch
    return self.create_batch(rest_items, pos=True)
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/dsindex.py", line 292, in create_batch
    batch = self.subset_by_pos(_batch_indices)
  File "/Users/Welcome/Documents/work/radio/radio/dataset/dataset/dsindex.py", line 80, in subset_by_pos
    return self.index[pos]
TypeError: only integer scalar arrays can be converted to a scalar index

Process finished with exit code 1

@Screenium Screenium mentioned this issue Mar 12, 2018
Closed
@roman-kh
Copy link
Member

Closing as non-active for a long time.
Feel free to reopen if the problem persists.

@lindehesse lindehesse mentioned this issue May 2, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@roman-kh @Screenium @AlexeyKozhevin