Skip to content

Missing Python 3 libs (with test) #312

stefanholek opened this Issue Aug 10, 2012 · 5 comments

2 participants



I see that #194 has been closed 8 months ago, but as of today virtualenv 1.7.2 does not install the required libs. In my case I am missing:


(No token and tokenize here, but they are mentioned in #194)

Due to the ImportErrors, it can happen that the TextIOWrapper for sys.stdout is not correctly set up during Python startup. The result is a sys.stdout with an encoding of 'ascii'.

I have also come up with a test which I will try to attach to this issue. The test fails when run in a virtualenv under Python 3.2.3. Someone in the know please put it in the right place (Alas, I failed to find the tests, is it all bas64?).


# This test fails when run in a virtualenv under Python 3.2.3

# Adding these libraries fixes the test:


import sys
import subprocess

def test_missing_libs_break_stdout_encoding_when_writing_to_pipe():
    # ImportErrors cause an unfortunate codepath to be taken in
    # textiowrapper_init(). See Modules/_io/textio.c:900
    # The result is a stdout encoding of 'ascii' for the subprocess
    process = subprocess.Popen(
        [sys.executable, '-c', 'import sys; print(sys.stdout.encoding)'],
    out, err = process.communicate()
    encoding = out.strip()

    if sys.version_info < (3,):
        # In Python 2 the encoding is always None
        assert encoding == 'None', \
            "%r != 'None'" % (encoding,)
        # In Python 3 the encoding should match the main
        # process' encoding
        encoding = encoding.decode('ascii')
        assert encoding == sys.stdout.encoding, \
            '%r != %r' % (encoding, sys.stdout.encoding)

if __name__ == '__main__':
carljm commented Aug 10, 2012

Thanks for the report. #194 was closed 8 months ago, and then very shortly reopened - it is currently open, and this is a duplicate of it.

Virtualenv's tests are not in, they are in the tests/ subdirectory of the repo. This test would need modification to become part of the test suite, as the test suite does not itself run within a virtualenv - the test would need to actually create the virtualenv.

I haven't played with this test yet, but I don't understand the concept behind it. If this test is run within a virtualenv, then shouldn't the "parent process" executing the test be subject to the exact same startup bug as the subprocess executed within the test? If so, why wouldn't the assertion always pass (as "ascii" == "ascii")? If not, why not? If the bug only shows up in the subprocess, not the parent process, despite the fact that both are running within the virtualenv, then it seems like something more complex is going on.

In any case it seems clear from this and #194 that the above modules ought to be included in REQUIRED_MODULES for Python 3.2.

@carljm carljm closed this Aug 10, 2012

Re the test: The difference is that the subprocess is writing to a pipe, whereas the main process writes to a tty.


Please also note that 'functools' is missing from the libraries installed under Python 3.3.0b1. All the others appear to be there.

carljm commented Aug 14, 2012

This was also fixed by 72dbd8b, which closed #194.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.