multiprocessing won't work with Tkinter (under Linux)

[akineko]

BPO	5527
Nosy	@asvetlov, @applio
Files	tk_test.py: A test program to reproduce the problem hanger.py: Minimal script to reproduce the bug in python 2 or 3

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-03-08.15:38:24.385>
created_at = <Date 2009-03-20.18:34:38.237>
labels = ['expert-tkinter', 'type-bug', 'library']
title = "multiprocessing won't work with Tkinter (under Linux)"
updated_at = <Date 2015-03-08.15:38:24.383>
user = 'https://bugs.python.org/akineko'

bugs.python.org fields:

activity = <Date 2015-03-08.15:38:24.383>
actor = 'davin'
assignee = 'none'
closed = True
closed_date = <Date 2015-03-08.15:38:24.385>
closer = 'davin'
components = ['Library (Lib)', 'Tkinter']
creation = <Date 2009-03-20.18:34:38.237>
creator = 'akineko'
dependencies = []
files = ['13383', '31222']
hgrepos = []
issue_num = 5527
keywords = []
message_count = 10.0
messages = ['83867', '85177', '85338', '85343', '85371', '126498', '194848', '195480', '202618', '237540']
nosy_count = 7.0
nosy_names = ['akineko', 'jahakala', 'asvetlov', 'pbwinston', 'sbt', 'James.Sanders', 'davin']
pr_nums = []
priority = 'normal'
resolution = 'third party'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue5527'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

[akineko]

Hello,

The attached test case, which uses multiprocessing module to run Tkinter
GUI process, runs flawlessly under Solaris but hung under Linux (CentOS5).

The test case is a trimmed version of much larger program but it still
exhibits the same problem.
I got a suggestion to use a function rather than a method of a class.
But it didn't make any difference.

I may have overlooked something but as far as I review my code, I
couldn't find anything that explains why the test case won't work (In
fact, it works under Solaris).

[jahakala]

The script tk_test.py produces the window with one button after one
removes the line 'from Tkinter import *' and adds line 'from Tkinter
import Tk, Button' inside Panel.draw() as a first line.

So importing Tkinter after the fork seems to solve the problem.
Importing it before the fork only makes the parent process consume more
memory anyway. Maybe importing the Tkinter (and thus _tkinter) causes
some initialisation to be done. If this should be done only in the
process that utilises Tkinter there might be problems like this.

[akineko]

Hello Jani Hakala,

Thank you very much for working on the case I created.
And, sorry for not getting back to you.

I have confirmed observation that the problem is fixed under Linux if
Tkinter is imported after fork().

However, this remains a problem for me and most other Tkinter programmers.
It is fairy common to import Tkinter using from Tkinter Import *.

This is because otherwise all Tk constants need to be prefixed with
Tkinter. That is very painful and I don't see such coding except small
examples.

If I import Tkinter module after fork(), I cannot use from Tkinter *.
Therefore this forces programmers to use Tkinter. prefix for every Tk
constants.

Is there any way to figure out the cause of this problem?
Or, is there any way to do "from Tkinter import *" after fork()?

Thank you for your help.

Aki-

[jahakala]

You can do something like
import gui
gui.start()
in your Panel.draw() and 'from Tkinter import *' in the gui module which
should contain your GUI-related code.

Or you could just do 'from Tkconstants import *' in your tk_test.py

[akineko]

Hello Jani Hakala,

Thank you for your suggestions.
Yes, from Tkconstants import * would ease the pain.
The second suggestion, importing gui in another file didn't work well
with my code as I'm using a class to host both gui process and non-gui
process such that separating gui into another class will create another
problem.

The first suggestion worked ... almost.
I needed to import a bunch of Tkinter related modules in not only gui()
but also other call back methods.
It solves the problem but the code became pretty messy.

It would be nice if the problem is solved so that I don't need to resort
to these workarounds. It is a bit frustrating as things work flawlessly
under Solaris.

I will play around trying to find a bit nicer workaround.

Thank you!
Aki-

[pbwinston]

We ran into this. Forking before importing Tkinter worked for us. We did the following which seems pretty clean:

main.py
import stdlib only
if __name__ == 'main':
<fork via multiprocessing.Process>
from application import App
App()

application.py
import Tkinter
class App...

[JamesSanders]

I recently got hit by this bug on 64-bit ubuntu 13.04, with python 3.3.1 and tcl/tk 8.5.13 installed from the Ubuntu repositories. However, I tried building the same versions of tcl, tk, and python locally, and couldn't reproduce the bug. I also built python 3.3.2 with several releases of tcl/tk. With 8.5.8 and earlier, I get a buffer overflow. With 8.5.9-14 and recent trunk versions of tk and tcl, everything works. With 8.6.0, I get the hang.

In the tcl/tk versions where the script works correctly, if I also have some tkinter code (such as instantiating a Tk and a Button) in the main process before the fork, then I sometimes get a crash with one of several different weird error messages from X, but this is very inconsistent.

The direct cause of the bug seems to be the call to Tcl_FindExecutable in _tkinter's initialization code, which tells tcl the path of the python executable, but as a side effect triggers tcl into running some initialization routines earlier than it would do otherwise. Removing this call (which doesn't seem to have any adverse effects on my system) appears to "fix" the bug, but if anything else triggers tcl's initialization code before the fork (such as instantiating a tkinter.Tk object), the hang returns.

I'm not really sure where else to go with this. I've attached a simplified python-3-compatible script displaying the problem in case someone else wants to have a look.

[JamesSanders]

I did a bit more digging and I think I've worked out what is going on. The particular bit of tcl initialization code that triggers the problem if it is run before the fork is Tcl_InitNotifier in tclUnixNotify.c. It turns out there is a known problem with this bit of tcl not being process-safe if tcl was built with threading support (this is discussed at https://bugs.archlinux.org/task/16401, for example).

The bug doesn't show up in my 8.5 builds as threading support is off by default, though the debian/ubuntu packages apparently have it switched on. Threading was turned on by default in 8.6, but a recent change to tclUnixNotify.c (discussed at https://issues.apache.org/bugzilla/show_bug.cgi?id=55153#c13 - it should be included in 8.6.1 and 8.5.15) appears to have fixed the whole problem anyway.

So hopefully the bug should disappear entirely in future releases of tcl, but for now you can work around it by building tcl without threads, calling exec in between the fork and any use of tkinter in the child process, or not importing tkinter until after the fork. I don't know if there should be a note about this somewhere in the tkinter docs?

[sbt]

So hopefully the bug should disappear entirely in future releases of tcl,
but for now you can work around it by building tcl without threads,
calling exec in between the fork and any use of tkinter in the child
process, or not importing tkinter until after the fork.

In 3.4 you can do this by using

    multiprocessing.set_start_method('spawn')

[applio]

Unable to reproduce on Ubuntu 12.04.5 with Python 2.7.8 and current libtk8.5 and libtcl8.5 releases using the attached hanger.py example.

Key findings to-date:

1. Posts to this issue have concluded that there exists a problem with thread- and process-safety in specific parts of Tcl library routines and that this known issue is addressed in more recent releases.
2. For those affected versions of the Tcl libraries, there is a workaround for in building those libraries to prevent the issue.
3. The Python Tkinter module (nor other standard library modules) does not appear to be a source of the described behaviors. Furthermore, Richard offers yet another workaround through multiprocessing's spawn.

The question was raised whether there should be a note added to the Tkinter module's docs about this situation. Because this issue is the result of particular versions on particular platforms built in a particular way and the issue appears addressed in current releases of those third party libraries, it probably makes good sense to track this sort of issue in the Tcl/Tk projects themselves and not in the Tkinter module's docs. If others encountering this issue in the future feel it is still too easy to end up with the right (wrong) combination of platform+tcl_version+build, they are encouraged to please open a new issue requesting a note be added to the documentation for Tkinter.

Closing this issue, marking it as "third party".