Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.5.2 crashers (from PyPy) #73069

Open
arigo mannequin opened this issue Dec 6, 2016 · 11 comments
Open

Python 3.5.2 crashers (from PyPy) #73069

arigo mannequin opened this issue Dec 6, 2016 · 11 comments
Labels
type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@arigo
Copy link
Mannequin

arigo mannequin commented Dec 6, 2016

BPO 28883
Nosy @brettcannon, @arigo, @vstinner, @serhiy-storchaka

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-12-06.11:49:47.741>
labels = ['type-crash']
title = 'Python 3.5.2 crashers (from PyPy)'
updated_at = <Date 2018-02-25.12:58:11.810>
user = 'https://github.com/arigo'

bugs.python.org fields:

activity = <Date 2018-02-25.12:58:11.810>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2016-12-06.11:49:47.741>
creator = 'arigo'
dependencies = []
files = []
hgrepos = []
issue_num = 28883
keywords = []
message_count = 9.0
messages = ['282518', '282519', '282520', '282521', '282522', '282523', '282525', '282564', '312809']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'arigo', 'vstinner', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue28883'
versions = ['Python 3.5']

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

As discussed on python-dev, I am creating omnibus issues from the lists of crashers, of wrong-according-to-the-docs, and of strange-behavior-only issues that I found while developing Python 3.5.2 support for PyPy. These occur with CPython 3.5.2 but most of them are likely still here in trunk.

This is the issue containing the crashers.

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

os.scandir() returns an iterable object that should not be used
from multiple threads. Doing so can e.g. cause one thread to
close the dirp while another thread is still using it. This is
likely to crash. Similarly, the test for (!iterator->dirp) at
the start of ScandirIterator_iternext() is only done once even
if the following loop runs two or three times because of "." or
".." entries.

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

(C2) os.scandir() direntry objects should not have stat() called from two
threads concurrently. It will make two stat objects and leak one of
them.

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

(C3) _PyGen_yf() checks the opcode at [f_lasti + 1], which is the next
opcode that will run when we resume the generator: either it is the
opcode following the YIELD, or it is exactly YIELD_FROM. It is not
possible at the moment to write Python code that compiles to a YIELD
immediately followed by YIELD_FROM, so by chance the two cases are
correctly distinguished. *However,* the discussion so far assumes
that the generator is not currently running. If it is (which probably
doesn't occur in reasonable Python code but can be constructed
manually), then this checks for example the byte/word that describes
the argument of the currently running opcode. If we're very unlucky
this byte has the value 72, which is YIELD_FROM. Total nonsense and
crashes follow.

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

(C4) faulthandler: register(): the signal handler, faulthandler_user(),
changes errno in faulthandler_dump_traceback() but fails to restore it
if chain=False. This can rarely cause random nonsense in the main
program.

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

(C5) setting f_lineno didn't evolve when the rest of the bytecodes evolved,
which means it is not safe any more::

    import sys

    def f():
        try:
            raise ValueError    # line 5
        except ValueError:
            print(42)           # line 7

    def my_trace(*args):
        print(args)
        if args[1] == 'line':
            f = args[0]
            if f.f_lineno == 5:
                f.f_lineno = 7
        return my_trace

    sys.settrace(my_trace)
    f()
    sys.settrace(None)

@arigo
Copy link
Mannequin Author

arigo mannequin commented Dec 6, 2016

(C6) I didn't try, but it seems that typeobject.c:mro_internal() is prone
to a refcount crash. It does this::

     old_mro = type->tp_mro;
     ...mro_invoke()...  /* might cause reentrance */
     type->tp_mro = new_mro;
     ...
     Py_XDECREF(old_mro);

This last XDECREF drops the reference held by the previous value of
type->tp_mro after we changed it. But type->tp_mro might have
changed because of mro_invoke(), which calls pure Python code. If it
did change, then old_mro is no longer the old value of
type->tp_mro. The wrong object gets decrefed.

@arigo arigo mannequin added the type-crash A hard crash of the interpreter, possibly with a core dump label Dec 6, 2016
@brettcannon
Copy link
Member

brettcannon commented Dec 6, 2016

Victor for C4.

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Feb 25, 2018

C5 is fixed in 3.8 by bpo-17611.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@iritkatriel
Copy link
Member

iritkatriel commented Nov 23, 2022

I thiink C6 was fixed in #66924.

@iritkatriel
Copy link
Member

iritkatriel commented Nov 23, 2022

@brandtbucher - is there any chance that C3(at #73069 (comment)) is still relevant?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Development

No branches or pull requests

3 participants