Opening a file holds the GIL when it calls "isatty()" #88385
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = None closed_at = <Date 2021-09-09.16:42:10.377> created_at = <Date 2021-05-23.17:18:13.788> labels = ['interpreter-core', '3.10', '3.9', 'type-crash', '3.11'] title = 'Opening a file holds the GIL when it calls "isatty()"' updated_at = <Date 2021-09-10.16:22:31.565> user = 'https://github.com/smurfix'
activity = <Date 2021-09-10.16:22:31.565> actor = 'lukasz.langa' assignee = 'none' closed = True closed_date = <Date 2021-09-09.16:42:10.377> closer = 'vstinner' components = ['Interpreter Core'] creation = <Date 2021-05-23.17:18:13.788> creator = 'smurfix' dependencies =  files = ['50270'] hgrepos =  issue_num = 44219 keywords = ['patch'] message_count = 12.0 messages = ['394208', '401261', '401402', '401450', '401458', '401470', '401478', '401495', '401497', '401584', '401588', '401591'] nosy_count = 5.0 nosy_names = ['vstinner', 'smurfix', 'lukasz.langa', 'vxgmichel', 'miss-islington'] pr_nums = ['28250', '28255', '28256', '28261', '28274', '28275'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'crash' url = 'https://bugs.python.org/issue44219' versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
The text was updated successfully, but these errors were encountered:
Opening a file calls
Please don't do that.
In my case, the file in question is implemented as a FUSE mount which is served by the same process (different thread of course). Thus holding the GIL at this point causes a rather interesting deadlock.
Tested with 3.9.
My team ran into this issue while developing a fuse application too.
In an effort to help this issue move forward, I tried to list all occurrences of the
9 of them are directly related to stdin/stdout/stderr, so it's probably not crucial to release the GIL for those occurrences:
Out of the remaining 4, only 1 releases the GIL:
Which gives 3 occurrences of non-stdstream specific usage of
The first one is used by
The second one is the one found by @smurfix and gets triggered when
The third one only triggers on windows when writing more than 32767 bytes to a file descriptor. A comment points to issue bpo-11395 (https://bugs.python.org/issue11395). Also, it seems from the function signature that this function might be called with or without the GIL held, which might cause the fix to be a bit more complicated than the first two use cases.
I hope this helps.
Please do. However, I do think that changing the stdstream related ioctl calls also is a good idea, if only for code regularity/completeness sake.
(Besides, nothing prevents somebody from starting a FUSE file system and then redirecting stdout to it …)
There are a couple of reasons why I did not make changes to the stdstream related functions.
The first one is that a PR with many changes is less likely to get reviewed and merged than a PR with fewer changes. The second one is that it's hard for me to make sure that those functions are always called with the GIL already held. Maybe some of them never hold the GIL in the first place, and I'm not familiar enough with the code base to tell.
So in the end it will probably be up to the coredev reviewing the PR, but better start small.
I ran some checks and
Since the change fixes a deadlock, I agree to backport it to 3.9 and 3.10.
Thanks for the fix!