Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

message_about_scripts_not_on_PATH does not expanduser() #6414

Closed
jflorian opened this issue Apr 16, 2019 · 12 comments
Closed

message_about_scripts_not_on_PATH does not expanduser() #6414

jflorian opened this issue Apr 16, 2019 · 12 comments

Comments

@jflorian
Copy link

@jflorian jflorian commented Apr 16, 2019

Environment

  • pip version: 18.1
  • Python version: 3.7
  • OS: Fedora 29

from python3-pip-18.1-1.fc29.noarch

Description
I was trying to install PyGitUp with pip3 install --user git-up and ran into an issue where I was warned:

  The script git-up is installed in '/home/jflorian/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 

I callled BS because I knew this was on my PATH. But then I noticed my PATH actually had ~/.local/bin and not /home/jflorian/.local/bin as claimed. I looked at wheel.py where the error originates and sure enough, there was no usage of os.path.expanduser(). I changed my PATH to have this already expanded and the error message disappeared.

Expected behavior
Most tools seem to know how to expand the tilde in my PATH, I would expect Pip to do so as well.

How to Reproduce

  1. export PATH with a local bin directory using a tilde, e.g., ~/.local/bin
  2. Install any package with scripts using the --user option.
  3. Observe the technically correct warning, but which is misleading w/o serious thought because it must be taken very literally and not as most users are likely to understand.
  4. Repeat with PATH having the tilde already expanded, e.g., /home/jflorian/.local/bin and observe the behavior I'd expect in step 3.
@pradyunsg pradyunsg added the triage label May 27, 2019
@chrahunt

This comment has been minimized.

Copy link
Member

@chrahunt chrahunt commented Jun 20, 2019

The purpose of this message is to help users identify why a particular script scriptname may not be available for execution as scriptname in their shell after installation of the containing package.

I tested several shells and it appears that only bash expands a literal ~ in PATH by default (or at least with the default options on my machine).

$ cat test.sh
#!/bin/sh

echo "#!/bin/sh
echo found
" > ~/example
chmod +x ~/example

for shell in /bin/bash /bin/zsh /usr/bin/fish /bin/csh /bin/dash /bin/ksh; do
    echo "$shell plain ~"
    HOME=~ PATH='~' $shell -c example
    echo "$shell expanded ~"
    HOME=~ PATH=~ $shell -c example
done
$ ./test.sh
/bin/bash plain ~
found
/bin/bash expanded ~
found
/bin/zsh plain ~
zsh:1: command not found: example
/bin/zsh expanded ~
found
/usr/bin/fish plain ~
fish: Unknown command 'example'
fish:
example
^
/usr/bin/fish expanded ~
found
/bin/csh plain ~
example: Command not found.
/bin/csh expanded ~
found
/bin/dash plain ~
/bin/dash: 1: example: not found
/bin/dash expanded ~
found
/bin/ksh plain ~
/bin/ksh: example: not found
/bin/ksh expanded ~
found

Versions:

ii  bash                                      4.4.18-2ubuntu1.1         amd64                     GNU Bourne Again SHell
ii  csh                                       20110502-3                amd64                     Shell with C-like syntax
ii  dash                                      0.5.8-2.10                amd64                     POSIX-compliant shell
ii  fish                                      2.7.1-3                   amd64                     friendly interactive shell
ii  ksh                                       93u+20120801-3.1ubuntu1   amd64                     Real, AT&T version of the Korn shell
ii  zsh                                       5.4.2-3ubuntu3.1          amd64                     shell with lots of features

As a result, if expanduser was used here then bash users would not tell the difference, but users of any other shell would not get the benefit of the warning.

@chrahunt

This comment has been minimized.

Copy link
Member

@chrahunt chrahunt commented Jul 25, 2019

Do we want to implement anything related to this or should we leave it as-is with the justification given above?

@pfmoore

This comment has been minimized.

Copy link
Member

@pfmoore pfmoore commented Jul 25, 2019

I think the current behaviour is fine. We don't want to go down the rabbit hole of trying to second-guess subtleties in the user's shell.

Note that the OP was able to interpret the message and fix the issue (and actually, could probably have left their PATH unchanged and simply ignored the warning) so there's not actually a failure here.

@jflorian

This comment has been minimized.

Copy link
Author

@jflorian jflorian commented Jul 25, 2019

OP here. I guess I shouldn't be surprised this is tied to a bashism. If you look at the issue linked in the description you can see that I did not have the option of just ignoring this message -- there was a subtle failure in that I couldn't use git up, but git-up was fine. I think leaving things here "as is" is suboptimal simply because many users aren't going to be able to figure this out like I did. Even if they can, that can take time that feels like an utter waste once the problem is understood.

What would be better? I'm not sure exactly. Could the test for existence actually be delegated to the user's shell? I mean the trouble here is that some Python code is trying to replicate what the shell is doing and if we're to avoid implementing various shell subtleties then why not go straight to the shell for the answer wrapping around something like test -x FOO after FOO has been installed. At the very minimum, an enhanced message that hints to this type of problem would be better than no action at all.

@pfmoore

This comment has been minimized.

Copy link
Member

@pfmoore pfmoore commented Jul 25, 2019

there was a subtle failure in that I couldn't use git up, but git-up was fine.

How is that related to the pip install? That sounds to me like the problem that pip was warning you about - that something couldn't correctly find the git-up command, because it didn't expand ~ in PATH. The comments in the linked issue suggest that git behaves the same as pip (and the other shells that @chrahunt checked) in not expanding ~, so it's bash that's the odd one out here.

I'm confused as to what you're asking for here. The message about "not on PATH" is a warning, so if it's wrong or misleading, you can ignore it (we could of course improve the wording if there's something we can agree on is better, but that still wouldn't change the behaviour). The fact that git up didn't work isn't a pip issue, so it's not something we can fix.

@chrahunt

This comment has been minimized.

Copy link
Member

@chrahunt chrahunt commented Aug 11, 2019

What if we warn about this case if we see a path in PATH that starts with ~? For example:

The script git-up is installed in '/home/jflorian/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
NOTE: The current PATH contains path(s) starting with `~`, which may not be expanded by all applications.
@chrahunt

This comment has been minimized.

Copy link
Member

@chrahunt chrahunt commented Aug 26, 2019

Assuming no other comments - the changes here would be:

  1. Update pip._internal.wheel.message_about_scripts_not_on_PATH here to check and trace a note as mentioned above if any path in PATH starts with a ~.
  2. Add a test to tests.unit.test_wheel.TestMessageAboutScriptsNotOnPATH here to cover this case.
@jflorian

This comment has been minimized.

Copy link
Author

@jflorian jflorian commented Aug 26, 2019

@chrahunt I like what you've proposed and am quite convinced that my time-to-grok the issue would have been significantly shortened had this been in place.

@mdebi

This comment has been minimized.

Copy link
Contributor

@mdebi mdebi commented Nov 11, 2019

Hi. I would like to work on this PR.

@chrahunt

This comment has been minimized.

Copy link
Member

@chrahunt chrahunt commented Nov 11, 2019

Hi @mdebi! My comment here should still be accurate. Please let us know if you have any questions while working on this.

@mdebi

This comment has been minimized.

Copy link
Contributor

@mdebi mdebi commented Nov 12, 2019

Please review and let me know your comments.

I was not sure what to pick as the extension for the news fragment and chose feature since this issue was marked as enhancement. Please let me know if the name for news file should be different. Thank you.

mdebi pushed a commit to mdebi/pip that referenced this issue Nov 13, 2019
@chrahunt

This comment has been minimized.

Copy link
Member

@chrahunt chrahunt commented Nov 16, 2019

Closed in #7345. Thanks @mdebi!

@chrahunt chrahunt closed this Nov 16, 2019
@lock lock bot added the S: auto-locked label Dec 16, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Dec 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.