Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cython crush in runtime using yield #3265

Closed
idokoren1 opened this issue Dec 9, 2019 · 2 comments · Fixed by #3268
Closed

cython crush in runtime using yield #3265

idokoren1 opened this issue Dec 9, 2019 · 2 comments · Fixed by #3268

Comments

@idokoren1
Copy link

Mac OS version 10.14
Python 3.7.3
using cython V0.29.1

The following code compile correctly. but crush in run time:

cython code:

    class GlobalFunctions:
        ITER_FUNC = None

    cdef class INSTANCE:
    
        def do_iter(self, *args, **kwargs):
            yield from GlobalFunctions.ITER_FUNC(*args, **kwargs)

python code:

def user_iter(x=1,b=None):
    for i in range(10):
        yield i

GlobalFunctions.ITER_FUNC = user_iter

t = INSTANCE()
for x in t.do_iter():
    print(x)
@da-woods
Copy link
Contributor

This is to do with generator functions that can receive keyword arguments but aren't passed them. Here's a much simpler example:

def f(**kwargs):
    yield 0
    
f()

It generates a segmentation fault when you import the module.

It crashes in the function __pyx_pf_13iterkwd_crash_f (iterkwd_crash is the module name):

__Pyx_INCREF(__pyx_cur_scope->__pyx_v_kwargs);

__kyx_v_kwargs is NULL. The function is called from __pyx_pw_13iterkwd_crash_1f where __pyx_v_kwargs is recognised as unused and set to 0.

@da-woods
Copy link
Contributor

I can get a few different variants of generated code depending on what I do with kwargs:

def g():
    return 0

def f(**kwargs):
    yield g(**kwargs)
    
f()

gives

if (__pyx_kwds) {
    __pyx_v_kwargs = PyDict_Copy(__pyx_kwds); if (unlikely(!__pyx_v_kwargs)) return NULL;
    __Pyx_GOTREF(__pyx_v_kwargs);
  } else {
    __pyx_v_kwargs = NULL;
  }

(i.e. kwargs is deliberately set to NULL if not passed, thus segfault).

In contrast

def f(**kwargs):
    yield from kwargs.keys()
    
f()

Doesn't crash, since kwargs is initialized to an empty dictionary if not passed

__pyx_v_kwargs = (__pyx_kwds) ? PyDict_Copy(__pyx_kwds) : PyDict_New(); if (unlikely(!__pyx_v_kwargs)) return NULL;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants