python hangs if import statement launches threads

[jeffstearns]

BPO	1175202
Nosy	@loewis
Files	bug1.py: Script demonstrating bug described here

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2005-04-03.16:24:35.000>
created_at = <Date 2005-04-02.05:24:51.000>
labels = ['interpreter-core']
title = 'python hangs if import statement launches threads'
updated_at = <Date 2005-04-03.16:24:35.000>
user = 'https://bugs.python.org/jeffstearns'

bugs.python.org fields:

activity = <Date 2005-04-03.16:24:35.000>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2005-04-02.05:24:51.000>
creator = 'jeffstearns'
dependencies = []
files = ['1657']
hgrepos = []
issue_num = 1175202
keywords = []
message_count = 2.0
messages = ['24856', '24857']
nosy_count = 2.0
nosy_names = ['loewis', 'jeffstearns']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1175202'
versions = ['Python 2.4']

[jeffstearns]

(This bug looks similar to bug 1175194, but reports a different
problem.)

I have a simple module which launches multiple HTTP client threads.

The main thread creates 10 HTTP clients, each of which fetches
documents from a web server. The main thread then simply goes to
sleep while the client threads work. The threads are launched when
the module is imported.

If I launch the script via
python bug1.py
it launches and runs OK.

But if I launch the script via
python -c 'import bug1'
it runs a bit, then hangs.

Here's an example:

jps@cure> ./python -c 'import bug1'
Using 10 threads
cccccccccc <- [program hangs here]

Each thread prints a character every time that it does something
interesting. The 'c' characters indicate that a connect syscall was
made. These should be followed by 'C', indicating the the connect
returned. That never happens.

You might argue that it's inappropriate to launch threads from within
import statement, but I can't find a specific prohibition against it.

Here's a trace of the last few syscalls before it hangs (pids are
actually thread ids):
[pid 15481] futex(0x820cc48, FUTEX_WAKE, 1) = 0
[pid 15481] futex(0x8148698, FUTEX_WAIT, 0, NULL <unfinished ...>
[pid 15482] futex(0x81a9a80, FUTEX_WAKE, 1) = 0
[pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0
[pid 15482] write(2, "c", 1c) = 1
[pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0
[pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0
[pid 15482] futex(0x8148698, FUTEX_WAIT, 0, NULL <- hangs here

Please note that this bug is similar to bug 1175194, but reports a
different problem.

[loewis]

Logged In: YES
user_id=21627

This is not a bug. The import statement returns when the
code of the module completes. In this example, the import
statement returns when the call

run(10)

completes. As the run method has a very long sleep, the
import won't return until that sleep is over. So the problem
is not that 10 threads are started, but that the module then
blocks in its module execution.

Furthermore, Python serializes all import statements, with a
single import lock. So while one thread performs an import,
other threads block when trying to import. So if the threads
just started try to import something, they wait until the
"main" import completes. If the main imports wait for the
threads to complete (i.e. the commented-out thread.join),
you get a dead-lock.

Closing as "won't fix".