Create venv with pip fails when target dir is under userappdata using Microsoft Store python

[AdamYoblick]

BPO	45337
Nosy	@pfmoore, @tjguk, @zware, @zooba, @miss-islington, @AdamYoblick
PRs	bpo-45337: Use the realpath of the new executable when creating a venv on Windows #28663 [3.10] bpo-45337: Use the realpath of the new executable when creating a venv on Windows (GH-28663) #28810 [3.9] bpo-45337: Use the realpath of the new executable when creating a venv on Windows (GH-28663) #28811

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-07.23:24:15.240>
created_at = <Date 2021-09-30.20:39:58.508>
labels = ['type-bug', '3.9', 'OS-windows']
title = 'Create venv with pip fails when target dir is under userappdata using Microsoft Store python'
updated_at = <Date 2021-10-07.23:24:15.240>
user = 'https://github.com/AdamYoblick'

bugs.python.org fields:

activity = <Date 2021-10-07.23:24:15.240>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-07.23:24:15.240>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2021-09-30.20:39:58.508>
creator = 'AdamYoblick'
dependencies = []
files = []
hgrepos = []
issue_num = 45337
keywords = ['patch']
message_count = 5.0
messages = ['402983', '402985', '403442', '403446', '403447']
nosy_count = 6.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'steve.dower', 'miss-islington', 'AdamYoblick']
pr_nums = ['28663', '28810', '28811']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45337'
versions = ['Python 3.9']

[AdamYoblick]

Repro steps:

1. Install Python 3.9 from the Microsoft Store
2. Try to create a virtual environment under the userappdata folder, using a command line similar to the following:

"C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2032.0_x64__qbz5n2kfra8p0\python3.9.exe" -m venv "C:\Users\advolker\AppData\Local\Microsoft\CookiecutterTools\env"

3. Observe the following error:

Error: Command '['C:\\Users\\advolker\\AppData\\Local\\Microsoft\\CookiecutterTools\\env\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 106.

Note that creating a venv without pip DOES work:

"C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2032.0_x64__qbz5n2kfra8p0\python3.9.exe" -m venv "C:\Users\advolker\AppData\Local\Microsoft\CookiecutterTools\env" --without-pip

BUT the venv is NOT at the specified location. This is because the Windows Store app creates a redirect when creating the venv, and that redirect is only visible from within the python executable.

This means that python doesn't respect the redirect when trying to install pip into the newly created venv.

[zooba]

Thanks Adam.

This analysis is correct, and I think there are two parts to this.

First, we probably need to add an os.path.realpath(path_to_venv) before we try and launch the environment. We *could* limit this to when it's under AppData, but I think limiting it to Windows is enough.

Second, if the realpath generated a different path, we should warn the caller. That wouldn't have helped in this (programmatic) case, but at least if someone is looking at the output they'll get an important hint.

(The docs describing this behaviour are at https://docs.microsoft.com/en-us/windows/msix/desktop/desktop-to-uwp-behind-the-scenes )

[zooba]

New changeset 6811fda by Steve Dower in branch 'main':
bpo-45337: Use the realpath of the new executable when creating a venv on Windows (GH-28663)
6811fda

[miss-islington]

New changeset eabca6e by Miss Islington (bot) in branch '3.10':
bpo-45337: Use the realpath of the new executable when creating a venv on Windows (GH-28663)
eabca6e

[miss-islington]

New changeset 06935bd by Miss Islington (bot) in branch '3.9':
bpo-45337: Use the realpath of the new executable when creating a venv on Windows (GH-28663)
06935bd