Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdout error at interpreter shutdown fails to return OS error status #49569

Closed
mkc mannequin opened this issue Feb 19, 2009 · 26 comments
Closed

stdout error at interpreter shutdown fails to return OS error status #49569

mkc mannequin opened this issue Feb 19, 2009 · 26 comments
Labels
interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@mkc
Copy link
Mannequin

mkc mannequin commented Feb 19, 2009

BPO 5319
Nosy @loewis, @birkenfeld, @terryjreedy, @pitrou, @rbtcollins, @vadmium, @zooba, @corona10
PRs
  • bpo-39828: Fix json.tool to catch BrokenPipeError. #18779
  • Files
  • devfull.patch
  • finalize-error.patch
  • finalize-error.v2.patch: Py_Finalize() returns -1
  • Py_FinalizeEx.patch: New API
  • Py_FinalizeEx-120.patch: Exit status 120
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2015-11-30.04:12:14.400>
    created_at = <Date 2009-02-19.17:38:00.077>
    labels = ['interpreter-core', 'type-bug']
    title = 'stdout error at interpreter shutdown fails to return OS error status'
    updated_at = <Date 2020-03-04.22:11:56.278>
    user = 'https://bugs.python.org/mkc'

    bugs.python.org fields:

    activity = <Date 2020-03-04.22:11:56.278>
    actor = 'corona10'
    assignee = 'none'
    closed = True
    closed_date = <Date 2015-11-30.04:12:14.400>
    closer = 'martin.panter'
    components = ['Interpreter Core']
    creation = <Date 2009-02-19.17:38:00.077>
    creator = 'mkc'
    dependencies = []
    files = ['18395', '37513', '39592', '39636', '40306']
    hgrepos = []
    issue_num = 5319
    keywords = ['patch']
    message_count = 26.0
    messages = ['82482', '82484', '112342', '112919', '112923', '113330', '148423', '232967', '244647', '244662', '244666', '244812', '244817', '244891', '248628', '249183', '249187', '249191', '249192', '249356', '249370', '249377', '249378', '255585', '255605', '255606']
    nosy_count = 11.0
    nosy_names = ['loewis', 'georg.brandl', 'terry.reedy', 'mkc', 'pitrou', 'rbcollins', 'Arfrever', 'python-dev', 'martin.panter', 'steve.dower', 'corona10']
    pr_nums = ['18779']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue5319'
    versions = ['Python 3.6']

    @mkc
    Copy link
    Mannequin Author

    mkc mannequin commented Feb 19, 2009

    $ python2.5 -c 'print((1, 2, 3))' > /dev/full || echo error status
    close failed: [Errno 28] No space left on device
    $

    The above sequence should also output 'error status' before the next
    prompt. That is, python should not be returning EXIT_SUCCESS in this
    situation.

    @mkc mkc mannequin added the type-bug An unexpected behavior, bug, or error label Feb 19, 2009
    @mkc
    Copy link
    Mannequin Author

    mkc mannequin commented Feb 19, 2009

    Oops, I should have used the old python print syntax. Nonetheless, the
    behavior is the same:

    $ python2.5 -c 'print 1, 2, 3' > /dev/full || echo error status
    close failed: [Errno 28] No space left on device
    $

    @mkc mkc mannequin added the interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) label Feb 19, 2009
    @birkenfeld
    Copy link
    Member

    birkenfeld commented Aug 1, 2010

    Python 2.6, from bpo-5320:

    $ python2.6 -c 'print 1, 2, 3' > /dev/full || echo error status
    close failed in file object destructor:
    Error in sys.excepthook:

    Original exception was:
    $

    Python 3.1 doesn't output anything and also doesn't set an error status.

    @terryjreedy
    Copy link
    Member

    terryjreedy commented Aug 4, 2010

    Georg, are you saying that there is or is not a problem with 2.6 (which is beyond non-critical bug fixes) and/or 3.1?

    @pitrou
    Copy link
    Member

    pitrou commented Aug 4, 2010

    Here is a possible patch for 3.x.
    It doesn't change the return status of the process, though.

    @pitrou
    Copy link
    Member

    pitrou commented Aug 8, 2010

    I've committed the py3k fix in r83854 (3.1 in r83855).
    As for 2.6/2.7, I've finally diagnosed the problem: 2.6 and upwards use PyErr_Print() in the file destructor, but the sys.stderr and sys.excepthook objects are already None at that time (because of interpreter shutdown). Therefore the error can't be printed out.
    2.5 and earlier used a direct fprintf() of the errno code to stderr.

    I propose to ignore the 2.6/2.7 issue of the error not being printed properly, since it only happens at shutdown with a misbehaving stdout, which is not very common.

    As for the issue of the stdout error not being reported in the process return code, this would be possible in 3.x, although it would involve changing the signature of Py_Finalize() to include a return status (it currently returns void).

    @pitrou pitrou changed the title I/O error during one-liner fails to return OS error status stdout error at interpreter shutdown fails to return OS error status Aug 8, 2010
    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 26, 2011

    New changeset 150e096095e5 by Antoine Pitrou in branch '3.2':
    Issue bpo-13444: When stdout has been closed explicitly, we should not attempt to flush it at shutdown and print an error.
    http://hg.python.org/cpython/rev/150e096095e5

    New changeset 37300a1df7d7 by Antoine Pitrou in branch 'default':
    Issue bpo-13444: When stdout has been closed explicitly, we should not attempt to flush it at shutdown and print an error.
    http://hg.python.org/cpython/rev/37300a1df7d7

    @vadmium
    Copy link
    Member

    vadmium commented Dec 20, 2014

    Here is a patch that changes Py_Finalize() to return -1 on error, and then sets the exit status to 1. It did not introduce any failures in the test suite for me. However I suspect it deserves more consideration about backwards compatibility etc, which is beyond my knowledge.

    @vadmium
    Copy link
    Member

    vadmium commented Jun 2, 2015

    Updated patch resolving minor merge conflicts with current code, a coding style fix, and some tweaks and syntax fixes to the documentation.

    @pitrou
    Copy link
    Member

    pitrou commented Jun 2, 2015

    I don't know how acceptable it is to add a return value to Py_Finalize(). Can it break the stable ABI?

    @zooba
    Copy link
    Member

    zooba commented Jun 2, 2015

    Going from void to int is fine for when the return value is ignored. However, those who check the return value will see garbage on earlier versions.

    As always, safest to add a new function here. Or make the stable version continue to be void - it won't affect the exports.

    @vadmium
    Copy link
    Member

    vadmium commented Jun 4, 2015

    I guess this would involve:

    • Making a new API called Py_Finalize2() or something that returns the status
    • Redefine the existing Py_Finalize() to call Py_Finalize2() and ignore the return value

    @pitrou
    Copy link
    Member

    pitrou commented Jun 4, 2015

    We would probably call it Py_FinalizeEx(), but yes.

    @vadmium
    Copy link
    Member

    vadmium commented Jun 6, 2015

    Here is a patch adding a new Py_FinalizeEx() API. Please let me know if anything else needs to be added. Perhaps some sort of Py_LIMITED_API condition, adding to PC/python3.def, etc. I am not really familiar with these things.

    I also changed the version notices to 3.6, assuming it is too late for 3.5.

    @rbtcollins
    Copy link
    Member

    rbtcollins commented Aug 15, 2015

    I think you need to make the following changes:

    • Py_Exit with a non-zero status code should probably preserve the code rather than replacing it with 1.
    • Ditto in Py_Main.
    • Add a defs entry for Py_FinalizeEx - should be a copy of the Py_Finalize entry with the return type modified.

    @vadmium
    Copy link
    Member

    vadmium commented Aug 26, 2015

    Thanks for the hints Robert. I will definitely add that defs entry.

    I’m a bit unsure about preserving an existing non-zero exit status. It seems a slightly more complicated and unexpected way of do it to me. But I cannot remember why I was originally interested in this issue. Do you have a use case where keeping the exit status > 1 would be good?

    Also need to add some entries to What’s New.

    @rbtcollins
    Copy link
    Member

    rbtcollins commented Aug 26, 2015

    def main():
       try:
           thing()
       except SomethingChanged:
           exit(1)
       except BinaryDiff:
           exit(2)
       except:
           exit(3)

    This is the sort of pattern e.g. 'diff' has.

    Should exit(3) there become exit(1) ? exit(1) means 'succeeded and there were changes'. So no, 3,for this app, means 'broke entirely'.

    Exiting with '1' is also poor/confusing for this app. So perhaps we should pick something rarely used - like 255. But anyhow the same logic applies: if an app signals 'error' with a different code, we shouldn't change the apps 'error' signal to be something else when we detect an error.

    But OTOH preserving a non-zero not-error code when an error occurs is also wrong.

    So I think, on reflection:

    1. pick an unusual code. Not 1. Not 77 (automake test skip code). Maybe 255? https://www.freebsd.org/cgi/man.cgi?query=sysexits&sektion=3&apropos=0&manpath=FreeBSD+10.2-RELEASE defines some codes we should avoid as well.
    2. We document that python itself may cause this exit code to be used and that apps written in python should never use that code
    3. we use your patch as it is - on shutdown error force it to this chosen value.

    What do you think?

    @vadmium
    Copy link
    Member

    vadmium commented Aug 26, 2015

    I guess I like setting the status to 1 because it matches the status already used by the default exception handling, the value when you raise SystemExit("error message"), and the value used for most internal errors. However I did discover that the exit status is 2 if you give Python wrong CLI arguments or there is an OS error opening a script file.

    I’m still rather unsure, only weakly in favour of my original proposal. Maybe see if anyone else has an opinion?

    @rbtcollins
    Copy link
    Member

    rbtcollins commented Aug 26, 2015

    Right. So I think one could also propose that the default exception handling should also change to this new number, but it is a different situation - programs don't conflict with that - they allow it to bubble up, but this situation is outside its control.

    CLI handling of Python itself is clearly outside both situations and there's no need to reevaluate it now.

    @vadmium
    Copy link
    Member

    vadmium commented Aug 30, 2015

    Maybe 255 is a reasonable choice. Another option is some value slightly less than 126, since Posix shells have special meanings for 126, 127, and > 128 values.

    My patches already document the exit status used under the Py_Exit() function: “If :c:func:`Py_FinalizeEx` indicates an error, the exit status is set to 1 [or 255].” Do you think it should be documented somewhere else as well, perhaps for the interpreter as a whole?

    @rbtcollins
    Copy link
    Member

    rbtcollins commented Aug 30, 2015

    It seems to me that how, and when, Python exits the process is important to folk who will never ever touch the C API.

    @vadmium
    Copy link
    Member

    vadmium commented Aug 31, 2015

    So far I haven’t noticed much documentation on the exit statuses generated by Python itself (e.g. for unhandled exceptions). I did find some discussion in <https://docs.python.org/3/library/sys.html#sys.exit\>, so perhaps I will add to that.

    Since it implies values beyond 127 may not work on some systems, I now propose to use the arbitrary value of 120 instead of 255. (Keeping clear of 126 and 127, which have a special meaning in Posix shells.) Although, that statement about 0–127 was added in 1998 (bbe92932a005), so may not be relevant these days.

    @vadmium
    Copy link
    Member

    vadmium commented Aug 31, 2015

    Here is a new patch setting the exit status to 120. I also added a What’s New entry, and some more documentation under sys.exit().

    Robert, I added an entry to /PC/python3.def, following the lead of your patch in bpo-2786. You mentioned modifying the return value, but in that file I don’t see any return values. Should I ignore that bit, or am I looking in the wrong place?

    @rbtcollins
    Copy link
    Member

    rbtcollins commented Nov 29, 2015

    @martin I was wrong re: the defs - they only cover function vs data, not return codes. So it looks fine to me.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 30, 2015

    New changeset 6b08429a3932 by Martin Panter in branch 'default':
    Issue bpo-5319: New Py_FinalizeEx() API to exit with status 120 on failure
    https://hg.python.org/cpython/rev/6b08429a3932

    @vadmium
    Copy link
    Member

    vadmium commented Nov 30, 2015

    Thanks everyone for helping to get this right.

    @vadmium vadmium closed this as completed Nov 30, 2015
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    interpreter-core Interpreter core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants