os.dup() creates an inheritable fd when handling a character file on Windows

[ZackerySpytz]

BPO	37267
Nosy	@pfmoore, @vstinner, @tjguk, @zware, @zooba, @MojoVampire, @ZackerySpytz
PRs	bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows #14051 [3.8] bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051) #14140 [3.7] bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051) #14141

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-17.08:49:27.884>
created_at = <Date 2019-06-13.10:12:30.664>
labels = ['type-bug', '3.8', '3.9', 'extension-modules', '3.7', 'OS-windows']
title = 'os.dup() creates an inheritable fd when handling a character file on Windows'
updated_at = <Date 2019-07-10.18:43:06.815>
user = 'https://github.com/ZackerySpytz'

bugs.python.org fields:

activity = <Date 2019-07-10.18:43:06.815>
actor = 'josh.r'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-17.08:49:27.884>
closer = 'vstinner'
components = ['Extension Modules', 'Windows']
creation = <Date 2019-06-13.10:12:30.664>
creator = 'ZackerySpytz'
dependencies = []
files = []
hgrepos = []
issue_num = 37267
keywords = ['patch']
message_count = 8.0
messages = ['345494', '345513', '345799', '345800', '345824', '345825', '345827', '347633']
nosy_count = 7.0
nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'zach.ware', 'steve.dower', 'josh.r', 'ZackerySpytz']
pr_nums = ['14051', '14140', '14141']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue37267'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[ZackerySpytz]

In PR 13739, Eryk Sun mentioned that the Windows implementation of os.dup() returns an inheritable fd when handling a character file. A comment in _Py_dup() makes it seem as though this is due to a belief that handles for character files cannot be made non-inheritable (which is wrong).

[vstinner]

If you care of inherited handles on Windows, please have a look at:

• bpo-18174: regrtest -R 3:3 checks for handle leak #7827 from https://bugs.python.org/issue18174
• bpo-33966, multiprocessing: Pool wait for worker start #7966 from https://bugs.python.org/issue33966

ref: https://pythondev.readthedocs.io/unstable_tests.html#windows-handles

I decided to give up because of multiprocessing race conditions. I failed to fix them.

[vstinner]

New changeset 28fca0c by Victor Stinner (Zackery Spytz) in branch 'master':
bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051)
28fca0c

[vstinner]

According to the discussion in PR 14051: if os.dup() fails to make the new file descriptor new inheritable for a character device, the error must not be ignored. Instead, the caller is supposed to use os.dup(fd, inheritable=False).

Before *this bugfix*, os.dup() didn't respect its contract: fd2 was inheritable sometimes. Now the caller is aware of such special case and so can handle it properly.

[vstinner]

New changeset 7158949 by Victor Stinner (Miss Islington (bot)) in branch '3.7':
bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051) (GH-14141)
7158949

[vstinner]

New changeset 693945d by Victor Stinner (Miss Islington (bot)) in branch '3.8':
bpo-37267: Do not check for FILE_TYPE_CHAR in os.dup() on Windows (GH-14051) (GH-14140)
693945d

[vstinner]

Thanks Zackery Spytz as usual!

[MojoVampire]

This may have caused a regression, see bpo-37549.