Interpreter does not terminate if daemonized while running multithreaded

[nikratio]

BPO	7931
Nosy	@loewis, @amauryfa
Files	test.py: Test Case

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/loewis'
closed_at = <Date 2010-02-16.17:53:33.270>
created_at = <Date 2010-02-14.23:35:13.496>
labels = ['interpreter-core', 'invalid', 'type-crash']
title = 'Interpreter does not terminate if daemonized while running multithreaded'
updated_at = <Date 2010-02-16.17:53:33.265>
user = 'https://bugs.python.org/nikratio'

bugs.python.org fields:

activity = <Date 2010-02-16.17:53:33.265>
actor = 'amaury.forgeotdarc'
assignee = 'loewis'
closed = True
closed_date = <Date 2010-02-16.17:53:33.270>
closer = 'amaury.forgeotdarc'
components = ['Interpreter Core']
creation = <Date 2010-02-14.23:35:13.496>
creator = 'nikratio'
dependencies = []
files = ['16228']
hgrepos = []
issue_num = 7931
keywords = []
message_count = 4.0
messages = ['99350', '99381', '99382', '99426']
nosy_count = 3.0
nosy_names = ['loewis', 'amaury.forgeotdarc', 'nikratio']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue7931'
versions = ['Python 2.6', 'Python 3.1']

[nikratio]

I'm observing a strange behaviour with a multithreaded program. After all threads have exited, the interpreter still hangs in PyThread_acquire_lock - but there should be no reason for hanging, because there is only one thread left:

(gdb) backtrace
#0 0x008f4422 in __kernel_vsyscall ()
#1 0x00c7ff75 in sem_wait@@GLIBC_2.1 ()
at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/sem_wait.S:80
#2 0x081047a8 in PyThread_acquire_lock (lock=0x9d7b090, waitflag=1)
at ../Python/thread_pthread.h:349
#3 0x0810794c in lock_PyThread_acquire_lock (self=0xb77cd480, args=0xb779c02c)
at ../Modules/threadmodule.c:46
#4 0x080dcef4 in call_function (f=0x9d7a004, throwflag=0) at ../Python/ceval.c:3706
#5 PyEval_EvalFrameEx (f=0x9d7a004, throwflag=0) at ../Python/ceval.c:2389
#6 0x080dddf2 in PyEval_EvalCodeEx (co=0xb76567b8, globals=0xb76c7c64, locals=0x0,
args=0x9d82e7c, argcount=1, kws=0x9d82e80, kwcount=0, defs=0xb7662538, defcount=1,
closure=0x0) at ../Python/ceval.c:2968
#7 0x080dc1b4 in fast_function (f=0x9d82d34, throwflag=0) at ../Python/ceval.c:3802
#8 call_function (f=0x9d82d34, throwflag=0) at ../Python/ceval.c:3727
#9 PyEval_EvalFrameEx (f=0x9d82d34, throwflag=0) at ../Python/ceval.c:2389
#10 0x080dddf2 in PyEval_EvalCodeEx (co=0xb765b2f0, globals=0xb76c7c64, locals=0x0,
args=0xb5400714, argcount=1, kws=0xb5400718, kwcount=0, defs=0xb7662678,
defcount=1, closure=0x0) at ../Python/ceval.c:2968
#11 0x080dc1b4 in fast_function (f=0xb54005d4, throwflag=0) at ../Python/ceval.c:3802
#12 call_function (f=0xb54005d4, throwflag=0) at ../Python/ceval.c:3727
#13 PyEval_EvalFrameEx (f=0xb54005d4, throwflag=0) at ../Python/ceval.c:2389
#14 0x080dddf2 in PyEval_EvalCodeEx (co=0xb765b920, globals=0xb76c7c64, locals=0x0,
args=0x9b16478, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at ../Python/ceval.c:2968
#15 0x0816014c in function_call (func=0xb766379c, arg=0x9b1646c, kw=0x0)
at ../Objects/funcobject.c:524
#16 0x0806120a in PyObject_Call (func=0xb766379c, arg=0x9b1646c, kw=0x0)
---Type <return> to continue, or q <return> to quit---
at ../Objects/abstract.c:2492
#17 0x080684ac in instancemethod_call (func=0xb77778c4, arg=0x9b1646c, kw=0x0)
at ../Objects/classobject.c:2579
#18 0x08062577 in PyObject_Call (func=0xb77778c4, arg=0x0, kw=<value optimized out>)
at ../Objects/abstract.c:2492
#19 0x08065a95 in call_function_tail (o=0xb779caac, name=0x8176b8b "_shutdown",
format=0x817a4dc "") at ../Objects/abstract.c:2524
#20 PyObject_CallMethod (o=0xb779caac, name=0x8176b8b "_shutdown", format=0x817a4dc "")
at ../Objects/abstract.c:2601
#21 0x0805c31b in WaitForThreadShutdown (argc=6, argv=0xbf8aa524)
at ../Modules/main.c:243
#22 Py_Main (argc=6, argv=0xbf8aa524) at ../Modules/main.c:623
#23 0x0805baeb in main (argc=6, argv=0xbf8aa524) at ../Modules/python.c:23

But:

(gdb) info threads

• 1 Thread 0xb77dc6c0 (LWP 7786) 0x008f4422 in __kernel_vsyscall ()

[nikratio]

Here is a short testcase that reproduces the problem. If you run the script it daemonizes correctly, but then the daemon does not terminate but hangs forever.

The crucial part seems to be to daemonize the program while there is more than one thread running.

[nikratio]

Reproduced with Python 3.1 as well.

[amauryfa]

libc.daemon() calls forks(), which duplicates only the main thread.
The other thread does not exists in the forked process, and the interpreter blocks while trying to wait for it...

Please use os.fork() instead, it has code to forget the threads created by the threading module (and clears some other locks as well, preventing deadlocks).
Or google for "daemonize python".