venv activate.bat is UTF-8 encoded but uses current console codepage #76590
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = 'https://github.com/zooba' closed_at = <Date 2019-07-26.16:39:28.243> created_at = <Date 2017-12-22.10:36:15.658> labels = ['type-bug', '3.8', 'OS-windows', '3.7', 'library', 'expert-unicode'] title = 'venv activate.bat is UTF-8 encoded but uses current console codepage' updated_at = <Date 2019-07-26.16:39:28.243> user = 'https://bugs.python.org/Jac0'
activity = <Date 2019-07-26.16:39:28.243> actor = 'steve.dower' assignee = 'steve.dower' closed = True closed_date = <Date 2019-07-26.16:39:28.243> closer = 'steve.dower' components = ['Library (Lib)', 'Unicode', 'Windows'] creation = <Date 2017-12-22.10:36:15.658> creator = 'Jac0' dependencies =  files =  hgrepos =  issue_num = 32409 keywords = ['patch'] message_count = 18.0 messages = ['308931', '308934', '312356', '312389', '312390', '312393', '329141', '329142', '329425', '329431', '329440', '329447', '329450', '329451', '329464', '329481', '332324', '348505'] nosy_count = 12.0 nosy_names = ['paul.moore', 'vinay.sajip', 'vstinner', 'tim.golden', 'ezio.melotti', 'zach.ware', 'eryksun', 'steve.dower', 'pablogsal', 'Jac0', 'miss-islington', 'Martin Bijl-Schwab'] pr_nums = ['5757', '5765', '5765', '5766', '10295', '10377', '10403', '10405', '8321'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue32409' versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']
The text was updated successfully, but these errors were encountered:
Let's say I have a folder c:\test-ä in Windows
Now if I run: py -m venv env
the result is incorrectly just: C:\Users\Username\AppData\Local\Programs\Python\Python36\python.exe
If I run: path
So clearly the encoding is broken for the folder name.
I can fix this by changing activate.bat character encoding to OEM-US and then replacing "test-├ż" by "test-ä".
If I now activate and run: where python
By running: path
So looks good here as well.
I suggest that what ever is creating activate.bat file, is using incorrect character encoding for the creation of the file. If this is somehow platform specific, there could be a guide in the venv documentation about how to fix this.
The CMD shell decodes batch scripts using the attached console's output codepage, which defaults to OEM. OTOH, venv writes the replacement values for the template activate.bat as UTF-8 (codepage 65001), which is correct and should not be downgraded to OEM.
Instead, the batch script could temporarily switch the console to codepage 65001. Then restore the previous codepage at the end. For example:
[rest of script]
This commit broke a lot of the Windows buildbots:
Can someone work on a patch soon? If not we need to revert the commits in all the branches that are failed to build :(
Pablo Galindo reverted the change because it broke Windows buildbots, and we have a policy to revert a change breaking buildbots if the regression cannot be fixed "quickly" event:
Is someone working on investigating the bug? Do you need help to reproduce the bug?
Copy of the test_venv error:
Traceback (most recent call last): File "D:\buildarea\3.x.bolen-windows8\build\lib\test\test_venv.py", line 302, in test_unicode_in_batch_file out, err = check_output( File "D:\buildarea\3.x.bolen-windows8\build\lib\test\test_venv.py", line 37, in check_output raise subprocess.CalledProcessError( TypeError: __init__() takes from 3 to 5 positional arguments but 6 were given
That error is a bug in the test, but it only shows up on an error path anyway. Without removing the extra None we don't get to see the actual error output.
I can't look into this over the next week or two, but a quick glance at the original PR looks like a lot of quotes are missing around executable paths, so maybe it was that?