Compatibility possible for numba jit compilation in Python runtime on GCP cloud functions?

[songololo]

Compatibility request for GCP cloud functions Python runtime

This is neither a feature request or a bug so much as request for compatibility with Google Cloud Platform's cloud functions. A corresponding issue has been logged with GCP.

At the moment numba doesn't work with GCP cloud functions' Python runtime. An example function and error message below. GCP cloud functions is able to install and import the numba package (e.g. printing numba.__version__ works) , but, numba is unable to jit compile a function, presumably something to do with how GCP hosts or virtualises its compute environments.

Is there a way to coordinate a solution with GCP as this would be an extremely useful use-case?

import numba

@numba.njit()
def sum_test_njit(a, b):
  return a + b

print(sum_test_njit(2, 3)) # should print '5'

Example error response from GCP cloud function Python 3.7 runtime:

severity:  "ERROR"   
 textPayload:  "Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions_v1beta2/worker.py", line 281, in run_background_function
    _function_handler.invoke_user_function(event_object)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions_v1beta2/worker.py", line 171, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions_v1beta2/worker.py", line 168, in call_user_function
    event_context.Context(**request_or_event.context))
  File "/user_code/main.py", line 25, in hello_pubsub
    print(f'numba working? {sum_test_njit(2, 3)}')
  File "/env/local/lib/python3.7/site-packages/numba/dispatcher.py", line 368, in _compile_for_args
    raise e
  File "/env/local/lib/python3.7/site-packages/numba/dispatcher.py", line 325, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/env/local/lib/python3.7/site-packages/numba/dispatcher.py", line 653, in compile
    cres = self._compiler.compile(args, return_type)
  File "/env/local/lib/python3.7/site-packages/numba/dispatcher.py", line 83, in compile
    pipeline_class=self.pipeline_class)
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 873, in compile_extra
    return pipeline.compile_extra(func)
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 367, in compile_extra
    return self._compile_bytecode()
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 804, in _compile_bytecode
    return self._compile_core()
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 791, in _compile_core
    res = pm.run(self.status)
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 253, in run
    raise patched_exception
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 245, in run
    stage()
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 678, in stage_nopython_backend
    self._backend(lowerfn, objectmode=False)
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 628, in _backend
    lowered = lowerfn()
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 615, in backend_nopython_mode
    self.flags)
  File "/env/local/lib/python3.7/site-packages/numba/compiler.py", line 991, in native_lowering_stage
    lower = lowering.Lower(targetctx, library, fndesc, interp)
  File "/env/local/lib/python3.7/site-packages/numba/lowering.py", line 93, in __init__
    self.env = Environment.from_fndesc(self.fndesc)
  File "/env/local/lib/python3.7/site-packages/numba/lowering.py", line 31, in from_fndesc
    mod = fndesc.lookup_module()
  File "/env/local/lib/python3.7/site-packages/numba/funcdesc.py", line 83, in lookup_module
    return sys.modules[self.modname]
KeyError: "Failed at nopython (nopython mode backend)\n'main'"

[stuartarchibald]

Thanks for the report. Is this still an issue with latest Numba (0.51.2)?

[esc]

Closing this issue due to lack of information on compatibility with the latest version of Numba. If more information becomes available please add it and reopen the issue. Thanks.