Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Segmentation fault with loop over memoryview followed by an exception #4662

Closed
da-woods opened this issue Feb 25, 2022 · 2 comments · Fixed by #4663
Closed

[BUG] Segmentation fault with loop over memoryview followed by an exception #4662

da-woods opened this issue Feb 25, 2022 · 2 comments · Fixed by #4663

Comments

@da-woods
Copy link
Contributor

da-woods commented Feb 25, 2022

Describe the bug

Iterating over a memoryview in a generator expression causes a crash (on cleanup I think). pandas-dev/pandas#46029 (comment)

To Reproduce
Code to reproduce the behaviour:

# testmviewbug.pyx
def f(const double[:] qs):
    for x in qs:
        pass
    raise ValueError()

Build it with cythonize -if testmviewbug.pyx

and run it in python to give:

>>> import testmviewbug
>>> import numpy as np
>>> testmviewbug.f(np.array([1.,2.,3.0]))
Fatal Python error: Acquisition count is -1 (line 3421)
Python runtime state: initialized
Traceback (most recent call last):
  File "testmviewbug.pyx", line 4, in testmviewbug.f
    raise ValueError()
ValueError
Aborted (core dumped)

Expected behavior
No crash

Environment (please complete the following information):

  • OS: [e.g. Linux, Windows, macOS]
  • Python version 3.8.12
  • Cython version current master

Additional context

Bisect says it started in ab1d728, which makes sense because it's a commit that'd significantly change what was generated.

@da-woods da-woods changed the title [BUG] Segmentation fault (possibly related to memoryviews in generator expression) [BUG] Segmentation fault (possibly related to memoryviews in list comprehensions) Feb 25, 2022
@da-woods
Copy link
Contributor Author

This only happens on the exception path. I think the issue is the line:

__PYX_XCLEAR_MEMVIEW(&__pyx_t_2, 1);

which only appears on the exception path and doesn't seem to be matched by an associated incref

@da-woods da-woods changed the title [BUG] Segmentation fault (possibly related to memoryviews in list comprehensions) [BUG] Segmentation fault with loop over memoryview followed by an exception Feb 25, 2022
@da-woods
Copy link
Contributor Author

I do have a fix for this which I'll submit in a bit (once I've got some proper tests for it)

da-woods added a commit to da-woods/cython that referenced this issue Feb 25, 2022
Fixes cython#4662

Memoryview temps weren't being properly incremented (at the start
of the loop) or decremented (at the end of the loop). They were
decremented in the exception handling case.

Long-term I think we really should unify reference counting for
memoryviews further. It's *slightly* different from PyObjects
(mainly that it happens in wrapper functions rather than the
main function) and that leads to a large chance that it just
gets missed.
scoder pushed a commit that referenced this issue Feb 26, 2022
Memoryview temps weren't being properly incremented (at the start
of the loop) or decremented (at the end of the loop). They were,
however, decremented in the exception handling case.

Long-term I think we really should unify reference counting for
memoryviews further. It's *slightly* different from PyObjects
(mainly that it happens in wrapper functions rather than the
main function) and that leads to a large chance that it just
gets missed.

Fixes #4662
@scoder scoder added this to the 3.0 milestone Feb 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants