Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Cythonizing fails due to AttributeError when calling a cpdef function from a cimport'ed module #4000

Closed
leofang opened this issue Feb 7, 2021 · 5 comments · Fixed by #4001

Comments

@leofang
Copy link

leofang commented Feb 7, 2021

Describe the bug
Not sure if this is related to #1317 or #3193. In Module 2 if I first hold a reference to a cpdef function from (cimported) Module 1, and then call that function (via the reference), the compiler errors out during cythonizing with the following output:

$ cythonize -i -3 test_cpdef_func_ptr2.pyx
Compiling /home/leofang/dev/test_cpdef_func_ptr2.pyx because it changed.
[1/1] Cythonizing /home/leofang/dev/test_cpdef_func_ptr2.pyx
Traceback (most recent call last):
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/bin/cythonize", line 11, in <module>
    sys.exit(main())
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Build/Cythonize.py", line 223, in main
    cython_compile(path, options)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Build/Cythonize.py", line 106, in cython_compile
    **options.options)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1102, in cythonize
    cythonize_one(*args)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1208, in cythonize_one
    result = compile_single(pyx_file, options, full_module_name=full_module_name)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 727, in compile_single
    return run_pipeline(source, options, full_module_name)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 515, in run_pipeline
    err, enddata = Pipeline.run_pipeline(pipeline, source)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 355, in run_pipeline
    data = run(phase, data)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 335, in run
    return phase(data)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 52, in generate_pyx_code_stage
    module_node.process_implementation(options, result)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/ModuleNode.py", line 143, in process_implementation
    self.generate_c_code(env, options, result)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/ModuleNode.py", line 385, in generate_c_code
    self.body.generate_function_definitions(env, code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 442, in generate_function_definitions
    stat.generate_function_definitions(env, code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 442, in generate_function_definitions
    stat.generate_function_definitions(env, code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 3174, in generate_function_definitions
    FuncDefNode.generate_function_definitions(self, env, code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 1981, in generate_function_definitions
    self.generate_function_body(env, code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 1743, in generate_function_body
    self.body.generate_execution_code(code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 448, in generate_execution_code
    stat.generate_execution_code(code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/Nodes.py", line 5988, in generate_execution_code
    value.generate_evaluation_code(code)
  File "/home/leofang/miniconda3/envs/cupy_cuda112_dev/lib/python3.7/site-packages/Cython/Compiler/ExprNodes.py", line 6052, in generate_evaluation_code
    if value and value.is_attribute and value.obj.type.is_pyobject:
AttributeError: 'NoneType' object has no attribute 'is_pyobject'

To Reproduce
Cythonizing the second module gives the above error:

test_cpdef_func_ptr1.pxd:

cpdef int f() except? -1

test_cpdef_func_ptr1.pyx:

cpdef int f() except? -1:
    return 123

test_cpdef_func_ptr2.pyx:

cimport test_cpdef_func_ptr1

cpdef g():
    x = test_cpdef_func_ptr1.f
    return x()

Expected behavior
If in the 2nd module I instead cimport only f from the 1st module and then do x = f, it'd work. I expect the same behavior for my above snippet.

Environment (please complete the following information):

  • OS: Linux
  • Python version: 3.7.9
  • Cython version: 0.29.21

Additional context
N/A

@leofang leofang changed the title [BUG] Got AttributeError when calling a cpdef function from a cimport'ed module [BUG] Cythonizing fails due to AttributeError when calling a cpdef function from a cimport'ed module Feb 7, 2021
@da-woods
Copy link
Contributor

da-woods commented Feb 7, 2021

This also breaks with cdef methods so it isn't cpdef special-casing

da-woods added a commit to da-woods/cython that referenced this issue Feb 7, 2021
@da-woods
Copy link
Contributor

da-woods commented Feb 7, 2021

It's easily enough fixed but even with that it isn't inferring the type correctly (thus it's always going through the Python call and you're getting no benefit from the cpdef)

@leofang
Copy link
Author

leofang commented Feb 8, 2021

It's easily enough fixed but even with that it isn't inferring the type correctly (thus it's always going through the Python call and you're getting no benefit from the cpdef)

Thanks for quick action @da-woods! Looks like you found a way to fix this type inference in #4001? Is there any way to work around this bug in current Cython releases to get the C function pointer directly?

@da-woods
Copy link
Contributor

da-woods commented Feb 8, 2021

Is there any way to work around this bug in current Cython releases to get the C function pointer directly?

Possibly using the &module.address? But that's untested

@leofang
Copy link
Author

leofang commented Feb 9, 2021

Ah...I see, thanks. I worked around by importing the function from a stub header.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants