PyTraceBack_Print() fails if signal received but PyErr_CheckSignals() not called

[sbt]

BPO	13673
Nosy	@amauryfa, @pitrou
Files	testsigint.zip traceback_checksignals.patch traceback_checksignals_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-12-28.21:34:59.902>
labels = ['interpreter-core', 'type-bug']
title = 'PyTraceBack_Print() fails if signal received but PyErr_CheckSignals() not called'
updated_at = <Date 2015-10-13.04:38:39.190>
user = 'https://bugs.python.org/sbt'

bugs.python.org fields:

activity = <Date 2015-10-13.04:38:39.190>
actor = 'martin.panter'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2011-12-28.21:34:59.902>
creator = 'sbt'
dependencies = []
files = ['24100', '24106', '24177']
hgrepos = []
issue_num = 13673
keywords = ['patch']
message_count = 7.0
messages = ['150319', '150328', '150331', '150340', '150342', '150352', '150875']
nosy_count = 3.0
nosy_names = ['amaury.forgeotdarc', 'pitrou', 'sbt']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13673'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[sbt]

If SIGINT arrives while a function implemented in C is executing, then it prevents the function from raising an exception unless the function first calls PyErr_CheckSignals(). (If the function returns an object (instead of NULL) then KeyboardInterrupt is raised as expected.)

For example, the following function just spins for 5 seconds before raising RuntimeError:

  static PyObject *
  testsigint_wait(PyObject *self, PyObject *arg)
  {
      clock_t start = clock();
      while (clock() - start < 5 * CLOCKS_PER_SEC) {
          /* pass */
      }
      //PyErr_CheckSignals();
      PyErr_SetNone(PyExc_RuntimeError);
      return NULL;
  }

If I call this function and press Ctrl-C before it completes, then I get the following:

  >>> import testsigint
  >>> a = testsigint.wait()
  ^C>>> print(a)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  NameError: name 'a' is not defined

So the call failed, but no exception was raised, and the variable "a" was not set!

I would have expected RuntimeError (or KeyboardInterrupt) to be raised. If I uncomment the PyErr_CheckSignals() line then I get RuntimeError as expected:

  >>> import testsigint
  >>> a = testsigint.wait()
  ^CTraceback (most recent call last):
    File "<stdin>", line 1, in <module>
  RuntimeError

Also, if I wrap the call in try...finally or try...except, I get a sensible "chained" traceback:

  >>> try:
  ...   testsigint.wait()
  ... finally:
  ...   print("done")
  ... 
  ^CTraceback (most recent call last):
    File "<stdin>", line 2, in <module>
  RuntimeError

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "<stdin>", line 2, in <module>
  KeyboardInterrupt

(Tested under Linux and Windows with the default branch.)

[sbt]

I have tried the same with Python 2.7.1 on Linux. The problem is the same, but one gets a partial traceback with no exception:

  >>> import sys, testsigint
  >>> testsigint.wait()
  ^CTraceback (most recent call last):
    File "<stdin>", line 1, in <module>

  >>> sys.last_value
  RuntimeError()

Both on 2.7 and 3.3 sys.last_value gives RuntimeError().

[sbt]

I think I have found the problem. PyTraceBack_Print() calls PyFile_WriteString(), which calls PyFile_WriteObject(), which calls PyObject_Str() which begins with

  PyObject_Str(PyObject *v)
  {
      PyObject *res;
      if (PyErr_CheckSignals())
          return NULL;
      ...

Since PyErr_CheckSignals() returns -1, PyTraceBack_Print() fails.

(Changed title.)

[sbt]

Attached is a patch for the default branch.

Before calling PyFile_WriteString() the patch saves the current exception. Then it calls PyErr_CheckSignals() and clears the current exception if any. After calling PyFile_WriteString() the exception is restored.

I am not sure this is an appropriate fix.

[pitrou]

I think calling PyErr_WriteUnraisable would be more appropriate than PyErr_Clear.
I also wonder whether it's ok to ignore the exception. Pressing e.g. Ctrl-C generally shouldn't fail to stop the program, even if another exception is being processed at that moment.
(of course, you could argue this is already the case when e.g. the signal is received while in a __del__)

[sbt]

I think calling PyErr_WriteUnraisable would be more appropriate than
PyErr_Clear.

You mean just adding

    PyErr_CheckSignals();
    if (PyErr_Occurred())
        PyErr_WriteUnraisable(NULL);

before the call to PyFile_WriteString()? That seems to work:

  >>> from testsigint import *; wait()
  ^CException KeyboardInterrupt ignored
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  RuntimeError

I also wonder whether it's ok to ignore the exception. Pressing e.g.
Ctrl-C generally shouldn't fail to stop the program, even if another
exception is being processed at that moment.

The ignoring and clearing of exceptions also happens higher (lower?) in the call stack in print_exception() and print_exception_recursive(). For example, print_exception() ends with

      /* If an error happened here, don't show it.
         XXX This is wrong, but too many callers rely on this behavior. */
      if (err != 0)
          PyErr_Clear();
  }

[sbt]

Trivial 3 lines patch.

I guess there is still a race: if Ctrl-C is pressed after PyErr_CheckSignals() is called but before PyObject_Str() then the printing of any exception can still be suppressed.