Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE ignores sys.excepthook in normal, subprocess mode #87174

Closed
Kenny2github mannequin opened this issue Jan 23, 2021 · 20 comments
Closed

IDLE ignores sys.excepthook in normal, subprocess mode #87174

Kenny2github mannequin opened this issue Jan 23, 2021 · 20 comments
Assignees
Labels
3.8 3.9 3.10 expert-IDLE type-bug

Comments

@Kenny2github
Copy link
Mannequin

@Kenny2github Kenny2github mannequin commented Jan 23, 2021

BPO 43008
Nosy @terryjreedy, @aroberge, @miss-islington, @Kenny2github
PRs
  • #24302
  • #24346
  • #24347
  • #24370
  • #24374
  • #24375
  • Files
  • bpo-43008.patch: Fix for this issue, patch file from GitHub PR
  • bpo-43008.patch.2: New patch with requested changes
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2021-01-27.00:27:44.633>
    created_at = <Date 2021-01-23.11:30:46.092>
    labels = ['expert-IDLE', 'type-bug', '3.8', '3.9', '3.10']
    title = 'IDLE ignores sys.excepthook in normal, subprocess mode'
    updated_at = <Date 2021-02-04.17:48:45.168>
    user = 'https://github.com/Kenny2github'

    bugs.python.org fields:

    activity = <Date 2021-02-04.17:48:45.168>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2021-01-27.00:27:44.633>
    closer = 'terry.reedy'
    components = ['IDLE']
    creation = <Date 2021-01-23.11:30:46.092>
    creator = 'AbyxDev'
    dependencies = []
    files = ['49757', '49758']
    hgrepos = []
    issue_num = 43008
    keywords = ['patch']
    message_count = 20.0
    messages = ['385529', '385540', '385542', '385547', '385548', '385550', '385560', '385561', '385719', '385741', '385742', '385743', '385744', '385745', '385748', '385924', '385925', '385926', '386475', '386487']
    nosy_count = 4.0
    nosy_names = ['terry.reedy', 'aroberge', 'miss-islington', 'AbyxDev']
    pr_nums = ['24302', '24346', '24347', '24370', '24374', '24375']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue43008'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @Kenny2github
    Copy link
    Mannequin Author

    @Kenny2github Kenny2github mannequin commented Jan 23, 2021

    Similar to https://bugs.python.org/issue12643 - IDLE ignores sys.excepthook and always prints exceptions.

    @Kenny2github Kenny2github mannequin added expert-IDLE type-bug 3.7 3.10 3.8 3.9 labels Jan 23, 2021
    @Kenny2github Kenny2github mannequin added expert-IDLE type-bug labels Jan 23, 2021
    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 23, 2021

    I looked at all 4 hooks.

    sys.breakpointhook is invoked by breakpoint() in user code, so is not IDLE's concern at present.

    sys.unraiseablehook is invoked by the interpreter when user code messes up badly, so is not IDLE's condern. (And it is an expert feature that beginners should ignore.)

    sys.displayhook is set to rpc.displayhook in run.py or, if started with -n no subprocess, pyshell.py. IDLE executes user code with exec(). Shell entries are compiled in 'single' mode. Exec invokes displayhook for expression statements so compiled. In both normal and -n mode, displayhook exceptions are displayed and processed as user code exceptions, with the last traceback line giving the bad line.

    sys.excepthook is invoked by the interpreter for uncaught exceptions.
    In IDLE's -n mode, sys.excepthook *is* invoked. If it raises, both tracebacks are printed, separated by "\nDuring handling of the above exception, another exception occurred:\n". This issue is about having the same effect in normal 2-process mode.

    Since, in the subprocess, IDLE catches all user exceptions for its custom handling, there are no uncaught user exceptions. I believe your patch calls the user hook in the proper place, but an uncaught exception in sys.excepthook would be nasty. Replace the call with something like

        try:
            sys.e...k(...)
        except BaseException as e:
            print_exception()
            print("\nDuring handling of the above exception, another exception occurred:\n", file=sys.stderr)
            self.usr_exc_info = sys.exc_info()
            print_exception()
    

    If possible, I want a unittest added in test_run.py. (I am not sure exactly how as I don't think we have yet directly faked receipt of compiled user code.)

    class Executive(unittest.TextCase):
    
        def test_good_excepthook(self):
    
        def test_bad_excepthook(self):

    A blurb is needed (and always is for anything non-trivial).

    In Doc/library/idle.rst, I want to insert " and reset display and exception handling" before the period ending the current Shell entry.

    Restart Shell
    Restart the shell to clean the environment.

    @terryjreedy terryjreedy removed the 3.7 label Jan 23, 2021
    @terryjreedy terryjreedy changed the title IDLE ignores sys.excepthook IDLE ignores sys.excepthook in normal, subprocess mode Jan 23, 2021
    @terryjreedy terryjreedy removed the 3.7 label Jan 23, 2021
    @terryjreedy terryjreedy changed the title IDLE ignores sys.excepthook IDLE ignores sys.excepthook in normal, subprocess mode Jan 23, 2021
    @Kenny2github
    Copy link
    Mannequin Author

    @Kenny2github Kenny2github mannequin commented Jan 23, 2021

    Made requested changes, besides the tests. I didn't manage to figure out how to go about testing an excepthook in the first place, let alone a good or bad one. Hope that's not an issue.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 23, 2021

    Andre: while I would like friendlier tracebacks available in IDLE without 3rd party installation, I doubt this will happen soon. In the meanwhile, I see that you are working to polish Friendly Traceback with a friendly contest. I presume FT uses sys.excepthook. If so, this issue is opportune for supporting your package. Feel free to test the minimal patch as is, or when fleshed out, and let us know how it works.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 23, 2021

    Ken: I will probably work on tests, perhaps as a followup or parallel issue. I will likely do a bit or refactoring to make testing easier.

    I just looked at print_exception and discovered that the double traceback can likely by handled by properly setting exc.context for the 2nd exception. But getting that right will be aided by having something that already works.

    Possible manual test protocol in Shell

    >> import sys
    >> def egood(a,b,c): print('Exception traceback', file=sys.stderr)

    >>> sys.excepthook = egood
    >>> 1/0
    # "Exception traceback"
    >>> def ebad(a,b,c): z
    
    >>> sys.excepthook = ebad
    >>> 1/0
    # Double traceback for ZeroDivisionError and NameError

    @aroberge
    Copy link
    Mannequin

    @aroberge aroberge mannequin commented Jan 23, 2021

    Terry: I will put it on my todo list.

    Friendly-traceback can currently work (indirectly) with IDLE; see https://aroberge.github.io/friendly-traceback-docs/docs/html/editor.html

    I am not sure what benefit this patch will yield when it comes to using Friendly-traceback with IDLE compared with what is currently possible.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 23, 2021

    Andre: with this patch, you could potentially add a function or submodule that would point sys.excepthook to a function that would print the exception and then interact before returning. If this worked, one could add, for instance, 'import friendly_traceback.idle' at the top of a file or at a Shell prompt and have friendly-traceback only activate when there is an exception. So it would work in Shell.

    I hope by 3.10 to have IDLE '>>>' prompts moved into a sidebar (instead of line numbers. F-T prompts would be distinctly different.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 23, 2021

    Ken, I read on another issue that an exception in an except suite is automatically linked to the caught exception, though the printed result may not be exactly the same. Once your CLA is 'activated', I will experiment with the options.

    @Kenny2github
    Copy link
    Mannequin Author

    @Kenny2github Kenny2github mannequin commented Jan 26, 2021

    CLA registers as signed now.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 26, 2021

    New changeset 7a34380 by Ken in branch 'master':
    bpo-43008: Make IDLE respect sys.excepthook (GH-24302)
    7a34380

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jan 27, 2021

    New changeset 68102fb by Miss Islington (bot) in branch '3.9':
    bpo-43008: Make IDLE respect sys.excepthook (GH-24302)
    68102fb

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jan 27, 2021

    New changeset 6f0346d by Miss Islington (bot) in branch '3.8':
    bpo-43008: Make IDLE respect sys.excepthook (GH-24302)
    6f0346d

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 27, 2021

    Ken, for next time -- there is a separate News category for IDLE.

    @Kenny2github
    Copy link
    Mannequin Author

    @Kenny2github Kenny2github mannequin commented Jan 27, 2021

    Oops, gotcha. Thanks for doing the tests. I'm a Python contributor! :D

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 27, 2021

    I hope this is not your last contribution. Another hint: if one uses bpo-12643 or bpo-12643, then the tracker makes the link -- and strikes it out if closed.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 29, 2021

    New changeset 11d75ec by Terry Jan Reedy in branch 'master':
    bpo-43008: Add 'Patch by Ken Hilton' (GH-24370)
    11d75ec

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jan 29, 2021

    New changeset cf88382 by Miss Islington (bot) in branch '3.8':
    bpo-43008: Add 'Patch by Ken Hilton' (GH-24370)
    cf88382

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 29, 2021

    New changeset bf782b2 by Miss Islington (bot) in branch '3.9':
    bpo-43008: Add 'Patch by Ken Hilton' (GH-24370) (bpo-24374)
    bf782b2

    @aroberge
    Copy link
    Mannequin

    @aroberge aroberge mannequin commented Feb 4, 2021

    Terry: as per your earlier comment, I have tried to use Friendly-traceback with the latest version. While it does set the exception hook correctly, it is not useful.

    Friendly-traceback (FT) needs to have access to the code that was executed. FT's REPL caches the code (similarly to what IPython does, but not CPython) prior to execution so that it can retrieve it correctly, have access to the relevant execution frame, etc. This is not possible when using IDLE's REPL. In fact, for some types of exceptions, when FT tries to get access to the code, it ends up triggering secondary exceptions.

    However, using the method describe in FT's documentation (with relies on FT's own console) still works correctly.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 4, 2021

    IDLE also caches shell input. Off topic here, I sent Andre an email about how FT could access it.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 3.9 3.10 expert-IDLE type-bug
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants