webbrowser.open on unix fails.

[ideasman42]

BPO	11432
Nosy	@birkenfeld, @terryjreedy, @gpshead, @pitrou, @ezio-melotti, @merwok, @ideasman42
Files	subprocess_same_fd.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2011-03-15.20:07:42.385>
created_at = <Date 2011-03-07.09:30:26.522>
labels = ['type-bug', 'library']
title = 'webbrowser.open on unix fails.'
updated_at = <Date 2011-03-19.05:17:12.215>
user = 'https://github.com/ideasman42'

bugs.python.org fields:

activity = <Date 2011-03-19.05:17:12.215>
actor = 'terry.reedy'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2011-03-15.20:07:42.385>
closer = 'gregory.p.smith'
components = ['Library (Lib)']
creation = <Date 2011-03-07.09:30:26.522>
creator = 'ideasman42'
dependencies = []
files = ['21053']
hgrepos = []
issue_num = 11432
keywords = ['patch']
message_count = 9.0
messages = ['130245', '130312', '130367', '130369', '130630', '130639', '131031', '131033', '131390']
nosy_count = 9.0
nosy_names = ['georg.brandl', 'terry.reedy', 'gregory.p.smith', 'pitrou', 'ezio.melotti', 'eric.araujo', 'ideasman42', 'neologix', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue11432'
versions = ['Python 3.2', 'Python 3.3']

[ideasman42]

On Linux - tested on: Arch linux @ Debian Squeeze, this fails
python -c "__import__('webbrowser').open('http://python.org')"

The exception thats raised is:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.2/webbrowser.py", line 70, in open_new_tab
    return open(url, 2)
  File "/usr/lib/python3.2/webbrowser.py", line 62, in open
    if browser.open(url, new, autoraise):
  File "/usr/lib/python3.2/webbrowser.py", line 276, in open
    success = self._invoke(args, True, autoraise)
  File "/usr/lib/python3.2/webbrowser.py", line 239, in _invoke
    stderr=inout, preexec_fn=setsid)
  File "/usr/lib/python3.2/subprocess.py", line 736, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.2/subprocess.py", line 1330, in _execute_child
    raise child_exception_type(errno_num, err_msg)
OSError: [Errno 9] Bad file descriptor

Noticed that this wont raise an error when stdin arg isn't passed to Popen:
line 237:
p = subprocess.Popen(cmdline, close_fds=True, stdin=inout,

Removing tdin=inout makes firefox load ok, however this is there for a reason so its not a useful fix ofcourse.

[neologix]

The problem lies here:

/* Close pipe fds. Make sure we don't close the same fd more than */
/* once, or standard fds. */
if (p2cread > 2) {
    POSIX_CALL(close(p2cread));
}
(c2pwrite > 2) {
    POSIX_CALL(close(c2pwrite));
}
if (errwrite != c2pwrite && errwrite > 2) {
    POSIX_CALL(close(errwrite));
}

If p2cread == c2pwrite (which is the case here since /dev/null is passed as stdin and stderr), we end up closing the same FD twice, hence the EBADF.
Just passing
-(c2pwrite > 2) {
+(c2pwrite > 2 && c2pwrite != p2cread) {
POSIX_CALL(close(c2pwrite));
}

Solves this (but you probably also want to check for (errwrite != p2cread) when closing c2pwrite).

Note that the Python implementation uses a set to avoid closing the same FD twice:

# Close pipe fds. Make sure we don't close the
# same fd more than once, or standard fds.
closed = set()
for fd in [p2cread, c2pwrite, errwrite]:
    if fd > 2 and fd not in closed:
        os.close(fd)
        closed.add(fd)

It might be cleaner to use a fd_set, i.e.:
fd_set set;
FD_ZERO(&set);
FD_SET(0, &set);
FD_SET(1, &set);
FD_SET(2, &set);
if (!FD_ISSET(p2cread, &set)) {
    POSIX_CALL(close(p2cread));
    FD_SET(p2cread, &fd);
}
if (!FD_ISSET(c2pwrite, &set)) {
    POSIX_CALL(close(c2pwrite));
    FD_SET(c2pwrite, &fd);
}
if (!FD_ISSET(errwrite, &set)) {
    POSIX_CALL(close(errwrite));
    FD_SET(errwrite, &fd);
}

But maybe it's just too much (and also, fd_set can be defined in different header files, and while I'm sure it's async-safe on Linux, I don't know if it's required as part of a standard...).

[neologix]

Attached is a patch checking that no FD is closed more once when
closing pipe FDs, along with an update for test_subprocess.

[pitrou]

That probably won't make a difference here, but be aware that we switched to Mercurial and the SVN repo won't get updates anymore.
See http://docs.python.org/devguide/ and http://docs.python.org/devguide/setup.html#getting-the-source-code for details.

[merwok]

Is this a webbrowser or subprocess bug? Please add the corresponding expert from http://docs.python.org/devguide/experts to the nosy list.

[gpshead]

subprocess in 3.2 bug from the looks of it. not sure if 2.7 or 3.1 are impacted at all, i'll remove them from the list after confirming.

[python-dev]

New changeset 08daf3ef6509 by Gregory P. Smith in branch 'default':
Add unittests demonstrating issue bpo-11432.
http://hg.python.org/cpython/rev/08daf3ef6509

New changeset adcf03b074b7 by Gregory P. Smith in branch 'default':
Fix issue bpo-11432. if the stdin pipe is the same file descriptor as either stdout or stderr
http://hg.python.org/cpython/rev/adcf03b074b7

New changeset ead52adcd0c8 by Gregory P. Smith in branch '3.2':
Fix issue bpo-11432. if the stdin pipe is the same file descriptor as either stdout or stderr
http://hg.python.org/cpython/rev/ead52adcd0c8

New changeset ad2bd8d338b0 by Gregory P. Smith in branch '3.2':
Add unittests demonstrating issue bpo-11432.
http://hg.python.org/cpython/rev/ad2bd8d338b0

[python-dev]

New changeset 82e010943c68 by Gregory P. Smith in branch '3.2':
bpo-11432 news entry.
http://hg.python.org/cpython/rev/82e010943c68

New changeset 8d3bcf57977b by Gregory P. Smith in branch 'default':
Misc/NEWS entry for bpo-11432
http://hg.python.org/cpython/rev/8d3bcf57977b

[terryjreedy]

With fix, test, and news in 3.2 and 3.3, is anything left to do?