Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

faulthandler should indicate if the fault happened in garbage collection #88632

Closed
maxballenger mannequin opened this issue Jun 20, 2021 · 7 comments
Closed

faulthandler should indicate if the fault happened in garbage collection #88632

maxballenger mannequin opened this issue Jun 20, 2021 · 7 comments
Labels
3.10 3.11 stdlib type-feature

Comments

@maxballenger
Copy link
Mannequin

@maxballenger maxballenger mannequin commented Jun 20, 2021

BPO 44466
Nosy @vstinner, @pablogsal, @maxballenger
PRs
  • #26823
  • #26826
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-06-21.12:38:14.279>
    created_at = <Date 2021-06-20.20:24:28.386>
    labels = ['type-feature', 'library', '3.10', '3.11']
    title = 'faulthandler should indicate if the fault happened in garbage collection'
    updated_at = <Date 2021-06-21.18:46:37.413>
    user = 'https://github.com/maxballenger'

    bugs.python.org fields:

    activity = <Date 2021-06-21.18:46:37.413>
    actor = 'maxballenger'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-06-21.12:38:14.279>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2021-06-20.20:24:28.386>
    creator = 'maxballenger'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44466
    keywords = ['patch']
    message_count = 7.0
    messages = ['396196', '396202', '396226', '396228', '396237', '396241', '396281']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'pablogsal', 'maxballenger']
    pr_nums = ['26823', '26826']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue44466'
    versions = ['Python 3.10', 'Python 3.11']

    @maxballenger
    Copy link
    Mannequin Author

    @maxballenger maxballenger mannequin commented Jun 20, 2021

    I have been working on debugging a segfault. When faulthandler catches the fault, it makes a printout like this:

    Current thread 0x00007f4fa62b2700 (most recent call first):
    File "/usr/lib/python3.6/site-packages/tornado/ioloop.py", line 919, in call_at
    File "/usr/lib/python3.6/site-packages/tornado/ioloop.py", line 502, in add_timeout
    ...

    However, when I run the same app with gdb, catch the segfault with gdb and and run py-bt, it makes a printout like this

    (gdb) py-bt
    Traceback (most recent call first):
      Garbage-collecting
      File "/usr/lib/python3.6/site-packages/tornado/ioloop.py", line 919, in call_at
        functools.partial(stack_context.wrap(callback), *args, **kwargs),
      File "/usr/lib/python3.6/site-packages/tornado/ioloop.py", line 502, in add_timeout
        return self.call_at(deadline, callback, *args, **kwargs)
      ...

    The important distinction here for me is the "Garbage-collecting" line. When debugging this issue with faulthandler, I thought that the segfault was happening somewhere in the execution stack of this ioloop.py function. It wasn't until I ran under gdb that I realized it was actually happening in garbage collection and more or less has nothing to do with ioloop.py. It seems like faulthandler should be able to tell that the segfault was actually generated in garbage collection and this would make faulthandler much more helpful for cases like this.

    Thank you for reading!

    @maxballenger maxballenger mannequin added stdlib type-feature labels Jun 20, 2021
    @pablogsal
    Copy link
    Member

    @pablogsal pablogsal commented Jun 20, 2021

    faulthandler has no access to the C-stack so it cannot see the gc_collect_main() as gdb does.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jun 21, 2021

    Maxwell: Oh, that's a good idea! I wrote PR 26823 to implement the feature. It's just 3 new lines in traceback.c :-)

            if (tstate == current_tstate && tstate->interp->gc.collecting) {
                PUTS(fd, "  Garbage-collecting\n");
            }

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jun 21, 2021

    New changeset d191639 by Victor Stinner in branch 'main':
    bpo-44466: Faulthandler now detects the GC (GH-26823)
    d191639

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jun 21, 2021

    New changeset 9b0bbb9 by Victor Stinner in branch '3.10':
    bpo-44466: Faulthandler now detects the GC (GH-26823) (GH-26826)
    9b0bbb9

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jun 21, 2021

    Maxwell: That was a nice idea and it's now added to Python 3.10 (will be part of the next Python 3.10 beta release)! I got an exception to land this feature in Python 3.10 even if we are past the feature freeze (see PR 26823 discussion).

    As explained in PR 26823, Py_FatalError() also gets this _Py_DumpTracebackThreads() enhancement ;-) See an example at:
    #26823 (comment)

    @maxballenger
    Copy link
    Mannequin Author

    @maxballenger maxballenger mannequin commented Jun 21, 2021

    Thank you Victor, sounds great!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 3.11 stdlib type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants