Tcl doesn't work inside a virtualenv on Windows #93

Closed
carljm opened this Issue Mar 15, 2011 · 16 comments

Projects

None yet
@carljm
Contributor
carljm commented Mar 15, 2011

Originally reported by Shawn Wheatley at https://bugs.launchpad.net/virtualenv/+bug/449537

Environment:
Tested on Windows XP SP3 and Windows 7
Python 2.6.3
Virtualenv 1.3.4

Within a fresh, just activated VirtualEnv, I get the following error trying to run a simple Tkinter window:

H:\My Documents\pythondev>virtualenv H:\testvirtualenv
New python executable in H:\testvirtualenv\Scripts\python.exe
Installing setuptools...................done.

H:\My Documents\pythondev>H:\testvirtualenv\Scripts\activate.bat
(testvirtualenv) H:\My Documents\pythondev>python
Python 2.6.3 (r263rc1:75186, Oct 2 2009, 20:40:30) [MSC v.1500 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.

import Tkinter
Tkinter._test()
Traceback (most recent call last):
File "", line 1, in
File "C:\Python26\Lib\lib-tk\Tkinter.py", line 3749, in _test
root = Tk()
File "C:\Python26\Lib\lib-tk\Tkinter.py", line 1643, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, want
objects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following directories:
C:/Python26/lib/tcl8.5 H:/testvirtualenv/lib/tcl8.5 H:/lib/tcl8.5 H:/testvir
tualenv/library H:/library H:/tcl8.5.2/library H:/tcl8.5.2/library

This probably means that Tcl wasn't installed properly.

Shawn Wheatley wrote on 2009-10-13:

Virtualenv does not copy the Tcl library file(s) (or any core C extensions, that I can see) to the new virtualenv, but instead adds a reference to the PYTHONPATH. If I copy the "tcl" folder from C:\Python26\ over to the root of the new Virtualenv, Tkinter.Tk() shows a new window without throwing an exception.
Jannis Leidel wrote on 2009-10-13: #2

Just suspecting here but could you create a virtualenv on the same drive as the system Python (C:)? I have the feeling it could be related to https://bugs.launchpad.net/virtualenv/+bug/352844.

Shawn Wheatley wrote on 2009-10-14:

Same problem when creating an environment on C:

C:\Temp>virtualenv TestTkinterBug
New python executable in TestTkinterBug\Scripts\python.exe
Installing setuptools...................done.

C:\Temp>cd TestTkinterBug

C:\Temp\TestTkinterBug>Scripts\activate.bat
(TestTkinterBug) C:\Temp\TestTkinterBug>python
Python 2.6.3 (r263rc1:75186, Oct 2 2009, 20:40:30) [MSC v.1500 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.

import Tkinter
Tkinter._test()
Traceback (most recent call last):
File "", line 1, in
File "C:\Python26\Lib\lib-tk\Tkinter.py", line 3749, in _test
root = Tk()
File "C:\Python26\Lib\lib-tk\Tkinter.py", line 1643, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, want
objects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following directories:
C:/Python26/lib/tcl8.5 C:/Temp/TestTkinterBug/lib/tcl8.5 C:/Temp/lib/tcl8.5
C:/Temp/TestTkinterBug/library C:/Temp/library C:/Temp/tcl8.5.2/library C:/tcl8.
5.2/library

This probably means that Tcl wasn't installed properly.

Shawn Wheatley wrote on 2009-10-15:

I've figured out the problem, but wondering if someone could give me a suggestion on a "proper" solution. I dug into the Python source and found that importing Tkinter on win32 causes an import of a module called FixTk:

http://svn.python.org/view/python/tags/r263/Lib/lib-tk/FixTk.py?revision=75184&view=markup

This file checks to see if the folder "tcl" exists in sys.prefix ("C:\python26\tcl"), or if the folder tcltk\lib exists in the parent of sys.prefix ("C:\tcltk\lib"). Once it finds an appropriate folder, it drills in to find the actual Tcl library folder, and sets appropriate environment variables (TCL_LIBRARY, TK_LIBRARY, TIX_LIBRARY). Since virtualenv doesn't copy the first folder to the new location, and the second folder doesn't exist (at least on my system), the rest of the module exits without completion--without setting any environment variables. You can see an example of this in a virtualenv on win32 by doing the following:

import os
os.environ.keys().index('TCL_LIBRARY')
Traceback (most recent call last):
File "", line 1, in
ValueError: list.index(x): x not in list
import Tkinter
os.environ.keys().index('TCL_LIBRARY')
Traceback (most recent call last):
File "", line 1, in
ValueError: list.index(x): x not in list

Compare this with stock Python on win32:

import os
os.environ.keys().index('TCL_LIBRARY')
Traceback (most recent call last):
File "", line 1, in
ValueError: list.index(x): x not in list
import Tkinter
os.environ.keys().index('TCL_LIBRARY')
26

I see a few ways to solve this problem, some more elegant than others:

  1. virtualenv copies the "tcl" folder on win32 to the target folder

Pros: the least impact on both virtualenv and the environment variables; this will continue to allow Tkinter to function as expected in the system Python.
Cons: feels kind of hacky compared to how virtualenv references the rest of the standard library

  1. virtualenv performs the functions of FixTk.py and sets the appropriate environment variables in activate.bat

Pros: this change is isolated to a Windows-only file (the batch file) and should also cause minimal "damage" in the case where the *_LIBRARY variables are already set (unless changing these in a batch file affects them system wide, I don't remember how exactly that works)
Cons: depending on how variables are set, this may destroy any system-wide *_LIBRARY variables associated with Tcl.

  1. something similar to VirtualEnvWrapper, set a post deploy hook that sets the appropriate variables

Pros: no patch to virtualenv (yay!)
Cons: Tkinter support on win32 under virtualenv is still broken, this is a workaround that would need to be applied separately on every fresh install (although I'm sure avid virtualenv users already have some bootstrap scripts they use in this fashion anyway)

I seem to be the only Tkinter/virtualenv user on Windows :) so I'd be happy to work on a patch, but I'd like some suggestions as to the approach.

Ian Bicking wrote on 2009-10-16: Re: [Bug 449537] Re: Tkinter calls fail in fresh VirtualEnv

On Thu, Oct 15, 2009 at 3:30 PM, Shawn Wheatley swheatley@gmail.com wrote:

  1. virtualenv performs the functions of FixTk.py and sets the
    appropriate environment variables in activate.bat

Pros: this change is isolated to a Windows-only file (the batch file) and should also cause minimal "damage" in the case where the *_LIBRARY variables are already set (unless changing these in a batch file affects them system wide, I don't remember how exactly that works)
Cons: depending on how variables are set, this may destroy any system-wide *_LIBRARY variables associated with Tcl.

This seems reasonable to me. I'm not clear if the same problem exists
on other platforms? It doesn't seem to for me.

After thinking a bit, I think the best option to implement this would
be to put the FixTk code into the virtualenv site.py. Then activation
won't be required to get Tkinter to work. The FixTk code would be
modified to use sys.real_prefix in addition to sys.prefix.

@whart222

Note that this issue blocks the use of MatPlotLib using a virtual Python environment. IMHO, this makes this ticket a big deal.

I like carljm's suggestion of putting the fix in site.py...

@JustinSGray

it would greatly help me if this was delt with. It is a big problem for using MatplotLib with the standard backend inside a virtual env.

@Wietse
Wietse commented Jan 10, 2012

I solved this by setting the following environment variables in my activate script:
$env:TCL_LIBRARY = "C:\Python32\tcl\tcl8.5"
$env:TK_LIBRARY = "C:\Python32\tcl\tk8.5"

@geographika

Thanks Wietse. I took a similar approach by modifying my activate.bat script to include:

set TCL_LIBRARY=C:\Python26\ArcGIS10.0\tcl\tcl8.5
set TK_LIBRARY=C:\Python26\ArcGIS10.0\tcl\tk8.5

mapplotlib is now working in the virtualenv.

@citterio citterio pushed a commit to citterio/virtualenv that referenced this issue Jun 29, 2014
Michele Citterio Fix Tcl inside a virtualenv on Windows (issue #93) 2232d33
@kp25
kp25 commented Jul 6, 2014

Modified the activate.bat script to include the above commands, and it is working.
I am using virtualenvwrapper-win and I had multiple virtualenvs and should i change the activate.bat in all of them..? or is there any one single specific file that would suffice.?

@citterio
citterio commented Jul 7, 2014

@kp25 - you could patch virtualenv applying #627, which solves the issue for all virtual envs you will create afterward. Otherwise, until that pull request is accepted you need to change your activate.bat file for every virtual env you create. The problem is that virtualenv does not copy to the virtual env the file that python needs in order to figure out how to import Tkinter.

You could also manually do for every one of your virtual envs what #627 does upon virtual env creation: add a copy of the file <your python install dir>\Lib\lib-tk\FixTk.py to <your virtualenv python dir>\Lib (don't create the lib-tk subdir), with lines 49 and 52 of this copied FixTk.py edited to replace sys.prefix with sys.real_prefix. The advantage of doing so over editing the activate.bat files is that you don't need to specify the correct path for your python install.

@pfmoore
Member
pfmoore commented Jul 7, 2014

@citterio I've added some comments to your PR. I don't use Tk at all myself, so I don't have any direct understanding of the situation, but I can understand the problem if this breaks matplotlib.

@rbtcollins
Contributor

Seems like FixTK could be fixed in the std lib itself?

@ghost
ghost commented Jun 16, 2015

for windows
in your dircetory "C:\Users\g2\Envs\DataVizproj\Scripts\activate.bat"
just add
set "TCL_LIBRARY=C:\Python27\tcl\tcl8.5"
set "TK_LIBRARY=C:\Python27\tcl\tk8.5"
and restart your cmd or shell
worked

@jedie
jedie commented Aug 19, 2015

Any news for a fix with Py2 and Py3 ?!?

EDIT:

set "TCL_LIBRARY=C:\Python27\tcl\tcl8.5"
set "TK_LIBRARY=C:\Python27\tcl\tk8.5"

But this will only work, if the version number are the same ;)

My work-a-round looks like this:

        if sys.version_info[0] == 2:
            virtualprefix = sys.prefix
            sys.prefix = sys.real_prefix
            import FixTk
            if "TCL_LIBRARY" not in os.environ:
                # reload module, so that sys.real_prefix be used
                reload(FixTk)
            sys.prefix = virtualprefix
        else: # Python 3
            virtualprefix = sys.base_prefix
            sys.base_prefix = sys.real_prefix
            from tkinter import _fix
            if "TCL_LIBRARY" not in os.environ:
                # reload module, so that sys.real_prefix be used
                from imp import reload
                reload(_fix)
            sys.base_prefix = virtualprefix

Any better idea?

EDIT: Full work-a-round here: jedie/DragonPy@0c1add8#diff-f11e22674d5a7f1c68650e7bdcea3a50R8

@jedie jedie added a commit to jedie/DragonPy that referenced this issue Aug 19, 2015
@jedie jedie add a "starter GUI" + work-a-round for tkinter usage with virtualenv …
…under windows

Squashed commit of the following:

commit e7beb32
Author: JensDiemer <git@jensdiemer.de>
Date:   Wed Aug 19 11:42:41 2015 +0200

    remove obsolete starter files

commit 5ed7406
Author: JensDiemer <git@jensdiemer.de>
Date:   Wed Aug 19 11:38:15 2015 +0200

    "starter GUI"

commit 0c1add8
Author: JensDiemer <git@jensdiemer.de>
Date:   Wed Aug 19 11:27:20 2015 +0200

    Add work-a-round for tkinter usage with virtualenv under windows,
see:

    pypa/virtualenv#93

commit a35a94c
Author: JensDiemer <git@jensdiemer.de>
Date:   Wed Aug 19 09:42:45 2015 +0200

    move settings in a frame, add status bar + code cleanup

commit e6fa73e
Author: JensDiemer <git@jensdiemer.de>
Date:   Tue Aug 18 23:06:12 2015 +0200

    WIP: seperate in frame/class

commit b7008d6
Author: JensDiemer <git@jensdiemer.de>
Date:   Tue Aug 18 22:06:48 2015 +0200

    replace .sh scripts

commit 3cecd83
Author: JensDiemer <git@jensdiemer.de>
Date:   Tue Aug 18 17:27:39 2015 +0200

    WIP: starter GUI
c6b0ae0
@jedie
jedie commented Aug 25, 2015

The same error with pypy-2.6.0... My work-a-round doesn't work here.
os.environ["TCL_LIBRARY"] is set to the right place (C:\\pypy-2.6.0-win32\\tcl\\tcl8.5) but maybe the pypy version will not use it?!?

I created: https://bitbucket.org/pypy/pypy/issues/2125/tcl-doesnt-work-inside-a-virtualenv-on too.

@dodgex
dodgex commented Oct 7, 2015

my workarround: copy tcl8.5 and tk8.5 to one of the paths mentioned in the error (one of those mentioned below this message Can't find a usable tk.tcl in the following directories:).

@ankostis ankostis added a commit to JRCSTU/allinone that referenced this issue Dec 18, 2015
@ankostis ankostis WorkAround pypa/virtualenv#93.
+ pip also snakemake.
6e8cfe5
@VelinGeorgiev

This is my workaround on Azure:
Go to your web app -> Configure -> app settings and set the env variables. You have to know where the python resides sys.real_prefix.
tkonazure

@techtonik techtonik added a commit to techtonik/virtualenv that referenced this issue Apr 5, 2016
@techtonik techtonik Fix issue #93 - Tcl doesn't work inside a virtualenv on Windows f55ad3f
@techtonik
Contributor

PR #888 fixed virtualenv with tcl/tk for me on both Python 2.7 and Python 3.5. Please confirm that it works on your configurations too.

@techtonik
Contributor

Fix merged.

@Ivoz Ivoz closed this Apr 18, 2016
@jayvdb
jayvdb commented Jul 21, 2016

Creating empty directories ...\tcl\tcl8.6 and ...\tcl\tk8.6 also works if you dont need Tk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment