Windows Store "stub" Python executables give confusing behaviour

[pfmoore]

BPO	41327
Nosy	@brettcannon, @pfmoore, @tjguk, @zware, @eryksun, @zooba, @uranusjr

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2020-07-17.18:50:03.779>
created_at = <Date 2020-07-17.18:42:52.443>
labels = ['3.9', 'OS-windows']
title = 'Windows Store "stub" Python executables give confusing behaviour'
updated_at = <Date 2020-07-21.17:23:05.470>
user = 'https://github.com/pfmoore'

bugs.python.org fields:

activity = <Date 2020-07-21.17:23:05.470>
actor = 'eryksun'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2020-07-17.18:50:03.779>
closer = 'brett.cannon'
components = ['Windows']
creation = <Date 2020-07-17.18:42:52.443>
creator = 'paul.moore'
dependencies = []
files = []
hgrepos = []
issue_num = 41327
keywords = []
message_count = 12.0
messages = ['373845', '373846', '373847', '373849', '373851', '374026', '374028', '374030', '374053', '374056', '374071', '374075']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'uranusjr']
pr_nums = []
priority = 'normal'
resolution = 'third party'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41327'
versions = ['Python 3.9']

[pfmoore]

First of all, I do know that this is an issue with the Windows Store distribution, rather than the python.org one. But (a) I don't know where to report a bug against the Store implementation except here, and (b) it's arguably a case of the Windows implementation "stealing" the Python command, so worth flagging against core Python.

The problem is that Windows 10 installs python and python3 executables by default, which open the Windows Store offering to install Python. That's not a bad thing - it's nice to have Python available with the OS! Although (see below) hijacking the python and python3 commands for this purpose has some problems.

But those stubs silently do nothing when run with command line arguments, and because the python.org distribution doesn't put Python on PATH by default, users end up with the stubs available even if they install python.org Python.

We're getting a lot of bug reports from users as a result of this, when they follow standard instructions like "execute python -m pip". With the stub on the path, this silently does nothing, even though the user has installed Python. This is a very bad user experience, and not one that projects like pip can do anything to alleviate, other than having extremely convoluted "how to run pip" commands:

"""
To run pip, type python -m pip if you're on Unix, and py -m pip on Windows. Unless you're using the Windows Store version when you should do python -m pip on Windows too. Or on Unix when you might need to do python3 -m pip. Or...
"""

I don't have a good answer to this issue. Maybe the Windows Store stubs could detect core Python and do something better if it's installed? Maybe the stubs should print an explanatory message if run with command line arguments? Maybe Store Python should be available on Windows via some less error-prone mechanism?

I'm pretty sure if a Linux distribution shipped a python command that didn't run Python the way users expected, there would be complaints. After all, we have PEP-394 that explicitly states how we expect Unix systems to work. Maybe we need something similar for Windows?

[pfmoore]

See, for example, pypa/packaging-problems#379 as an illustration of the user (and project maintainer!) confusion this causes.

[brettcannon]

Closing as "third-party" as those stubs are controlled by Microsoft Windows and has nothing to do with us as a project beyond that they install the Windows Store copy that Steve uploads (although this is all feedback to Steve who has connections on the Windows team).

And I will say I believe there are shims on some Linux distros like Ubuntu for things like venv, pip, etc. which are not included in-box which people typically expect to be there.

[pfmoore]

I thought that might be the answer. But does anyone know where I can repost this as an issue against the Store distribution? I'm happy to report there if I can find out how...

Hopefully Steve can point me in the right direction.

[uranusjr]

FWIW, I tweeted about this a while ago (January) and IIRC Steve said there’s plan to change that. https://twitter.com/uranusjr/status/1212450480917340160

[zooba]

It already returns a non-zero exit code (should be (IIRC) 9009 to match the built-in cmd.exe result), and I've been trying to get the message added for at least a year now.

Unfortunately, I can only push it so far before it has to work through the Windows team's process, and then there's nobody to push it along. All that could really be done from outside is to organise a "why don't you fix the dev experience" spam campaign, but I'm not going to condone that ;)

[uranusjr]

What would be the best channel to raise this issue to the Windows team from the outside? It does not need to be a spam campaign, but it’d be nice if we could direct the affected users somewhere instead of pypa/packaging-problems and various issue trackers, where the Windows team wouldn’t see.

[zooba]

It's already gone through the correct channels, so any other submissions will be duped by the triagers.

The best person to post at is me, but I've suffered enough for it :) I'm pushing.

[pfmoore]

Understood that these would be duplicates, and I certainly don't want to put more pressure on you, but I think there would still be value in having a known reporting channel for people to say "I am also affected by this":

• MS Developers get a more objective feel for the scale of the issue.
• Users who report the issue can track progress (they will presumably get told *what* their report is a duplicate of, and can follow the underlying issue).
• Python package developers have something definite that they can offer to users.

My suspicion here is that MS simply doesn't have a user-visible support channel for the Store Python stubs, which is why we've ended up discussing this here.

[eryksun]

there would still be value in having a known reporting channel

Windows 10 has a "Feedback Hub" to report problems and search for existing feedback that's similar. You could report a problem with the "AppInstaller" app. In this case, the app execution alias targets "AppInstallerPythonRedirector.exe". This is a GUI program, so CMD won't wait on it and set %errorlevel% if run from an interactive prompt -- not unless one uses start /w to force it to wait. (Sidebar: CMD determines that it's a GUI program by querying the PEB address and reading the process PEB via ReadProcessMemory... hack-o-rama.)

IMO, the installer-redirector in this case should be a console application that prints a message to stdout to inform the user that it's opening the store to install Python. If it fails, it should print an error message to stderr and return a non-zero exit status.

[zooba]

Windows 10 has a "Feedback Hub" to report problems and search for existing feedback that's similar.

This is the correct channel, but ultimately the "real" issue tracker is private, so all you'll ever get is "not finished yet" that suddenly becomes "finished" (and generally that's updated on release, not on commit, because we've found that users get confused when something is claimed to be fixed but they're still seeing the issue).

IMO, the installer-redirector in this case should be a console application that prints a message to stdout to inform the user that it's opening the store to install Python.

This is exactly what it does. The fact that it is SUBSYSTEM:WINDOWS instead of CONSOLE is what I'm trying to get fixed. It's not complicated, it's just not something that anyone can force through besides the team that owns the code, and they own a lot of other things that need fixing as well (and the release schedule doesn't allow for "just one little fix"). Shipping an OS is *hard* ;)

[eryksun]

This is exactly what it does. The fact that it is SUBSYSTEM:WINDOWS

Sorry, I neglected to check:

C:\\\>python3 script.py 2\>&1 | more
Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640