Python doesn't exit with proper resultcode on SIGINT in runpy (pymain_run_module)

[graingert]

BPO	41602
Nosy	@gvanrossum, @gpshead, @ncoghlan, @ambv, @eryksun, @graingert
PRs	bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module #21956 [3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) #22397 [3.8] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) #22398
Files	test_exit.py test_exit.txt test_exit_runpy.py test_exit_command_stdin.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-09-22.15:53:35.762>
created_at = <Date 2020-08-20.19:58:37.660>
labels = ['interpreter-core', 'type-bug', '3.8', '3.9', '3.10']
title = "Python doesn't exit with proper resultcode on SIGINT in runpy (pymain_run_module)"
updated_at = <Date 2020-10-05.16:10:20.261>
user = 'https://github.com/graingert'

bugs.python.org fields:

activity = <Date 2020-10-05.16:10:20.261>
actor = 'lukasz.langa'
assignee = 'none'
closed = True
closed_date = <Date 2020-09-22.15:53:35.762>
closer = 'gvanrossum'
components = ['Interpreter Core']
creation = <Date 2020-08-20.19:58:37.660>
creator = 'graingert'
dependencies = []
files = ['49414', '49415', '49416', '49417']
hgrepos = []
issue_num = 41602
keywords = ['patch']
message_count = 22.0
messages = ['375732', '375733', '375735', '375736', '375775', '375898', '376984', '377000', '377012', '377013', '377014', '377019', '377020', '377021', '377022', '377024', '377027', '377028', '377339', '377447', '377451', '378057']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'gregory.p.smith', 'ncoghlan', 'lukasz.langa', 'eryksun', 'graingert']
pr_nums = ['21956', '22397', '22398']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41602'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[graingert]

when running "python -m ham.py" KeyboardInterrupt should result in -2, but results in 1

see test case

[graingert]

see also https://bugs.python.org/issue1054041

[graingert]

I think I've eliminated runpy.py, as I still get a -2 when using:

import runpy
runpy.run_module("ham")

or

runpy._run_module_as_main("ham")

see attached test_exit_runpy.py

It seems the only difference is if pymain_run_file or pymain_run_module is used.

[graingert]

I've now eliminated pymain_run_stdin and pymain_run_command, both process a KeyboardInterrupt as -2

see attached test_exit_command_stdin.py

[graingert]

adding vstinner, as this seems related to pymain_run_module rather than runpy

[graingert]

I've confirmed that the pymain_run_module fails on master in the same way with Mac, Windows and Ubuntu

[graingert]

adding __main__ owners to nosy

[gvanrossum]

Thomas, can you explain the problem in English? Those elaborate test programs don’t do it for me, sorry.

[graingert]

The return code of python on linux when the program is ended with a KeyboardInterrupt should be -2, this works in most cases - except when called via "python -m"

Does this repl help?

Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib, subprocess
>>> pathlib.Path("spam.py").write_text("raise KeyboardInterrupt")
23
>>> subprocess.run(["python3", "spam.py"])
Traceback (most recent call last):
  File "spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', 'spam.py'], returncode=-2)
>>> subprocess.run(["python3", "-m", "spam"])
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/graingert/projects/spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', '-m', 'spam'], returncode=1)
>>>

[gvanrossum]

Okay, I got it. Now my next question. What do you mean by "I've eliminated runpy" and various other remarks along those lines? Have you actually identified the root cause of the problem? What's your suggested fix (without test framework).

[eryksun]

The return code of python on linux when the program is ended with
a KeyboardInterrupt should be -2

In general, the exit status for an unhandled KeyboardInterrupt (i.e. _Py_UnhandledKeyboardInterrupt) should be the same as the default SIGINT handler. This is implemented by exit_sigint in Modules/main.c. In POSIX, it uses the actual default SIGINT handler via kill(). In Windows, it uses the exit status that the default console control handler would use, STATUS_CONTROL_C_EXIT (0xC000_013A).

[graingert]

Okay, I got it. Now my next question. What do you mean by "I've eliminated runpy" and various other remarks along those lines?

I can use runpy._run_module_as_main from another Python entry, eg -c and get the correct -2

>>> subprocess.run(["python3", "-c", "__import__('runpy')._run_module_as_main('spam')"])
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/graingert/projects/osirium-main/spam.py", line 1, in <module>
    raise KeyboardInterrupt
KeyboardInterrupt
CompletedProcess(args=['python3', '-c', "__import__('runpy')._run_module_as_main('spam')"], returncode=-2)

Have you actually identified the root cause of the problem? What's your suggested fix (without test framework).

No I don't have a fix

[gvanrossum]

Okay, so according to eryksun the fix requires C code. @graingert, are you interested in developing such a fix? Or @eryksun do you have an idea on how to fix it?

[graingert]

I'm assuming _Py_UnhandledKeyboardInterrupt is getting incorrectly cleared somewhere in here

cpython/Modules/main.c

Lines 291 to 300 in fc23a94

	Py_DECREF(runpy);
	Py_DECREF(runmodule);
	Py_DECREF(module);
	Py_DECREF(runargs);
	if (result == NULL) {
	return pymain_exit_err_print();
	}
	Py_DECREF(result);
	return 0;
	}

    Py_DECREF(runpy);
    Py_DECREF(runmodule);
    Py_DECREF(module);
    Py_DECREF(runargs);
    if (result == NULL) {
        return pymain_exit_err_print();
    }
    Py_DECREF(result);
    return 0;
}

[gvanrossum]

Thanks, I think this is the fix:

diff --git a/Modules/main.c b/Modules/main.c
index 4a76f4461b..3d1bbee3a0 100644
--- a/Modules/main.c
+++ b/Modules/main.c
@@ -288,6 +288,10 @@ pymain_run_module(const wchar_t *modname, int set_argv0)
         return pymain_exit_err_print();
     }
     result = PyObject_Call(runmodule, runargs, NULL);
+    if (!result && PyErr_Occurred() == PyExc_KeyboardInterrupt) {
+        _Py_UnhandledKeyboardInterrupt = 1;
+    }
+
     Py_DECREF(runpy);
     Py_DECREF(runmodule);
     Py_DECREF(module);

Can you submit it as a PR? I don't have the time (but I can review once you've got tests working etc.)

[graingert]

I pushed that patch

[gvanrossum]

Why did you remove 3.10?

[graingert]

nice the tests pass with that fix

[gvanrossum]

New changeset a68a2ad by Thomas Grainger in branch 'master':
bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (bpo-21956)
a68a2ad

[ambv]

New changeset ca8d46d by Łukasz Langa in branch '3.9':
[3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (bpo-22397)
ca8d46d

[ambv]

New changeset ae46229 by Thomas Grainger in branch '3.8':
[3.8] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (bpo-22398)
ae46229

[ambv]

New changeset c26a666 by Łukasz Langa in branch '3.9':
[3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (bpo-22397)
c26a666