Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

segfault with fused types and __kwdefaults__ #1470

zpincus opened this issue Sep 25, 2016 · 5 comments

segfault with fused types and __kwdefaults__ #1470

zpincus opened this issue Sep 25, 2016 · 5 comments


Copy link

zpincus commented Sep 25, 2016

The below segfault occurs with version 0.25a0, on Python 3.5.2. I have verified it on Mac OS X 10.11.6 and on Linux (kernel 4.7.2, x86_64).

It is not a recent regression -- I reported this to the cython-devel email list in March, when I had observed the bug with Python 3.4.3 / Cython 0.23.4 and Python 2.7.10 / Cython 0.22.1 . At the time, there was no reply. When I saw that 0.25a0 was out, I thought I ought to verify that this is still an issue and post to github now that cython is using that.

Anyhow, the issue is that a cython function with a fused type and a keyword argument will cause an immediate segfault when its __kwdefaults__ attribute is looked up. (Which happens on any sort of introspection, such looking up the function docstring in ipython, or even tab completion of the __kwdefaults__ attribute name in plain python.)

The below is the simplest example:
Create the file foo.pyx:

ctypedef fused FUSED:

def foo(FUSED bar, baz=None):

Then in Python:

>>> import pyximport; pyximport.install()
(None, <pyximport.pyximport.PyxImporter object at 0x10f7c0410>)
>>> import foo
zsh: segmentation fault

This works of course when compiled with a script as well; the above is just the most minimal test case. There's no segfault without the keyword argument, and no segfault if the fused type is replaced with a regular type.

A relevant chunk of backtrace from Linux is below. The backtrace on the mac is identical.

Program received signal SIGSEGV, Segmentation fault.
__pyx_pf_3foo_12__defaults__ (__pyx_self=0x7ffff414d528)
    at /home/zplab/.pyxbld/temp.linux-x86_64-3.5/pyrex/foo.c:1506
1506      __Pyx_INCREF(__Pyx_CyFunction_Defaults(__pyx_defaults2, __pyx_self)->__pyx_arg_baz);
(gdb) bt
#0  __pyx_pf_3foo_12__defaults__ (__pyx_self=0x7ffff414d528)
    at /home/zplab/.pyxbld/temp.linux-x86_64-3.5/pyrex/foo.c:1506
#1  0x00007ffff3228e51 in __Pyx_CyFunction_init_defaults (op=0x7ffff414d528)
    at /home/zplab/.pyxbld/temp.linux-x86_64-3.5/pyrex/foo.c:3136
#2  __Pyx_CyFunction_get_kwdefaults (op=0x7ffff414d528)
    at /home/zplab/.pyxbld/temp.linux-x86_64-3.5/pyrex/foo.c:3206
#3  0x00007ffff79a9bb3 in _PyObject_GenericGetAttrWithDict (
    obj=0x7ffff414d528, name=0x7ffff69ffcf0, dict=0x0) at Objects/object.c:1059
#4  0x00007ffff7a26c8e in PyEval_EvalFrameEx (f=<optimized out>, 
    throwflag=<optimized out>) at Python/ceval.c:2743
#5  0x00007ffff7a2eb49 in _PyEval_EvalCodeWithName (_co=<optimized out>, 
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>, 
    argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, 
    closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4018
#6  0x00007ffff7a2ecd8 in PyEval_EvalCodeEx (_co=<optimized out>, 
    globals=<optimized out>, locals=<optimized out>, args=<optimized out>, 
    argcount=<optimized out>, kws=<optimized out>, kwcount=0, defs=0x0, 
    defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:4039
Copy link

hchapman commented Apr 5, 2018

I just ran into this bug myself. Cython 0.28.1 installed from pip just moments ago on Python 3.6.3.

Copy link

hchapman commented Apr 5, 2018

Subscripting works. For the original MWE, calling help(foo[float]) doesn't die, and foo[float].__kwdefaults__ returns None. (No subscript -> segfault as usual).

I'm super naive, but looking at the code it seems that an InitDefaults call is missing for the fused, subscriptable type. Namely, the part surrounded by ">>>>" for a specialization does not appear for the final generic part at the location "<<<<"...

  __pyx_t_12 = __pyx_FusedFunction_NewEx(&__pyx_fuse_1__pyx_mdef_9pd_markov_6markov_35foo, 0, __pyx_n_s_foo, NULL, __pyx_n_s_pd_markov_markov, __pyx_d, ((PyObject *)__pyx_codeobj__31)); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 385, __pyx_L1_error)
  if (!__Pyx_CyFunction_InitDefaults(__pyx_t_12, sizeof(__pyx_defaults21), 1)) __PYX_ERR(0, 385, __pyx_L1_error)
  __Pyx_CyFunction_Defaults(__pyx_defaults21, __pyx_t_12)->__pyx_arg_baz = Py_None;
  __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_12, __pyx_t_7);
  __Pyx_CyFunction_SetDefaultsGetter(__pyx_t_12, __pyx_pf_9pd_markov_6markov_80__defaults__);
  if (PyDict_SetItem(__pyx_t_11, __pyx_n_s_int, __pyx_t_12) < 0) __PYX_ERR(0, 385, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0;
  __pyx_t_12 = __pyx_FusedFunction_NewEx(&__pyx_mdef_9pd_markov_6markov_7foo, 0, __pyx_n_s_foo, NULL, __pyx_n_s_pd_markov_markov, __pyx_d, ((PyObject *)__pyx_codeobj__31)); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 385, __pyx_L1_error)
  __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_12, __pyx_t_7);
  __Pyx_CyFunction_SetDefaultsGetter(__pyx_t_12, __pyx_pf_9pd_markov_6markov_78__defaults__);
  ((__pyx_FusedFunctionObject *) __pyx_t_12)->__signatures__ = __pyx_t_11;
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_foo, __pyx_t_12) < 0) __PYX_ERR(0, 385, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0;
  __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0;

I've tried looking at the actual Cython code that generates this, but I'm a bit out in the deep end on this one.

Copy link

hchapman commented Apr 5, 2018

The issue appears to be that in FusedNode.FusedCFuncDefNode.analyse_expressions() the node at self.resulting_fused_function.arg does not populate a defaults parameter, even though all of the specialized nodes in self.specialized_pyfuncs do.

I've tried messing around (I even just set the defaults manually to be the same as self.specialized_pyfuncs[0]) and I just do not know enough about the inner workings of the Compiler to get things to work.

Copy link

scoder commented Apr 8, 2018

Thanks for investigating. Yes, the problem is that the defaults of the dispatcher function do not get initialised. And it seems that that would also not be trivial, because - what should the defaults be? They might differ across the specialised functions depending on their concrete argument types.

So, I guess, the best (or at least easiest) solution for now would be to make sure things don't crash, but assume there are no default arguments for the dispatch function. Or inherit them from the first specialisation, whatever proves to be simpler to implement.

Copy link

artemru commented Feb 14, 2019

The problem is reproducible with py3.7 and cython=0.29.5. Is there any known workaround ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet

No branches or pull requests

4 participants