Errors in site.py not reported properly

[Rhamphoryncus]

BPO	1771260
Nosy	@gvanrossum, @tiran

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2007-10-30.18:56:38.220>
created_at = <Date 2007-08-09.21:37:51.000>
labels = []
title = 'Errors in site.py not reported properly'
updated_at = <Date 2008-01-06.22:29:46.734>
user = 'https://bugs.python.org/Rhamphoryncus'

bugs.python.org fields:

activity = <Date 2008-01-06.22:29:46.734>
actor = 'admin'
assignee = 'none'
closed = True
closed_date = <Date 2007-10-30.18:56:38.220>
closer = 'gvanrossum'
components = ['None']
creation = <Date 2007-08-09.21:37:51.000>
creator = 'Rhamphoryncus'
dependencies = []
files = []
hgrepos = []
issue_num = 1771260
keywords = []
message_count = 4.0
messages = ['32615', '32616', '32617', '56978']
nosy_count = 3.0
nosy_names = ['gvanrossum', 'Rhamphoryncus', 'christian.heimes']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1771260'
versions = ['Python 3.0']

[Rhamphoryncus]

(Ignore the p3yk dir name. This has been updated to the py3k branch.)

rhamph@factor:~/src/python-p3yk/build$ make
object :
type : TypeError
refcount: 4
address : 0x8239f0c
lost sys.stderr
make: *** [sharedmods] Error 1

The root cause is that (due to some local modifications) site.py failed to load and gave an error. This can be easily duplicated by opening up Lib/site.py:main and putting 1/0 on the first line.

However, the ZeroDivisionError that should cause never gets printed. Python/pythonrun.c:initsite attempts to retrieve sys.stderr, which fails because site.py never got a chance to install it (!), then passes the NULL file object pointer to PyFile_WriteString, which turns that into a new exception (replacing the old one). initsite ignores the return value indicating the exception, instead clearing it, and the interpreter continues to load, no one the wiser.

Several other exceptions may happen and get squashed, I'm not sure.

Eventually, Python/sysmodule.c:sys_excepthook calls Python/pythonrun.c:PyErr_Display, which attempts to retrieve sys.stderr, and failing that calls _PyObject_Dump() on the exception (explaining the weird message). Oddly, there's a *second* if statement, which then prints the "lost sys.stderr" line.

Possible remedies:

1. PyErr_Display's dump message is not very clear.
2. initsite should go directly to stderr if it can't retrieve sys.stderr. Alternatively, since site.py is now more important, it could be changed into a fatal error. Yet another option is to explicitly check for sys.stderr even on success and make that alone into a fatal error.
3. The error printing APIs could be modified to internalize the stderr retrieval. Upon failure they could print a brief "stderr unavailable; original exception was ..." message.

[Rhamphoryncus]

Following up on the possible remedies, I'm thinking PySys_WriteStderr could be extended to take file object as the first argument, and if null it writes to low-level stderr instead. That way the callers can add the message about "lost sys.stderr" themselves, rather than it being interspersed in their output.

[tiran]

I run into trouble with stderr myself and I agree that the problem should be resolved somehow by falling back to the file descriptor 2 (stderr). The current implementation makes it unnecessarily hard to debug problems in site.py, io.py and possible also codecs.*.

[gvanrossum]

Fixed by Christian Heimes's patch, see bpo-1352 and bpo-1329.