Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip fails to install remote dependency when it is a .whl and follows PEP 440 #6202

Closed
vinicyusmacedo opened this issue Jan 25, 2019 · 9 comments · Fixed by #6203

Comments

@vinicyusmacedo
Copy link
Contributor

@vinicyusmacedo vinicyusmacedo commented Jan 25, 2019

Environment

  • pip version: pip 18.1
  • Python version: Python 2.7.12
  • OS: Ubuntu 16.04.1 - Linux 4.15.0-42-generic

I'm using virtualenv on the version 15.2.0

Description

When following PEP 508 and PEP 440's direct references and adding a dependency to a file, say requirements.txt like this:

boto @ https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl

doing a pip install -r requirements.txt gives this error.

Requirement u'boto @ git+https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl' looks like a filename, but the file does not exist
Processing ./boto @ git+https:/files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/home/vinicyusmacedo/Documents/Development/myapp/boto @ git+https:/files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl'

Expected behavior

Pip should download and install the dependency.

pip install -r tests.txt
Collecting boto@ https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl# from https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl# (from -r tests.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl
Installing collected packages: boto
Successfully installed boto-2.49.0

Note that including a # to the end of the URL works just fine.

How to Reproduce

  1. Create a file (say requirements.txt) with some URLs to remote .whl files following PEP 440's direct references
  2. Run pip install -r requirements.txt
  3. An error occurs.
@vinicyusmacedo

This comment has been minimized.

Copy link
Contributor Author

@vinicyusmacedo vinicyusmacedo commented Jan 25, 2019

Some thoughts

Is this some expected behavior? That what the PEP 440 says:

Public index servers SHOULD NOT allow the use of direct references in uploaded distributions. Direct references are intended as a tool for software integrators rather than publishers.

So, I believe I shouldn't be referencing a .whl file from somewhere like pypi.org?
Does that apply to private pypi repositories?

If that's not the case, I noticed that is_url on download.py simply splits an URL and uses the part before a : as the scheme. On the PEP 440 examples, the URL includes the package name @ the URL. So the scheme won't fall in any of the expected schemes.

def is_url(name):
    # type: (Union[str, Text]) -> bool
    """Returns true if the name looks like a URL"""
    if ':' not in name:
        return False
    scheme = url.split(':', 1)[0].lower()
    return scheme in ['http', 'https', 'file', 'ftp'] + vcs.all_schemes
@uranusjr

This comment has been minimized.

Copy link
Member

@uranusjr uranusjr commented Feb 5, 2019

I believe it is allowed to point a requirement to a file on the index server; what PEP 440 says is that the server itself should not allow it. The problem here is more likely that pip’s name parser is not smart enough to do the right thing.

@vinicyusmacedo

This comment has been minimized.

Copy link
Contributor Author

@vinicyusmacedo vinicyusmacedo commented Feb 5, 2019

@uranusjr got it, it is not up to pip to make such restriction.

@cjerdonek

This comment has been minimized.

Copy link
Member

@cjerdonek cjerdonek commented Mar 3, 2019

@uranusjr Quick question: is this issue related to your comment here: #6097 (comment)? If so, how is it related?

@vinicyusmacedo

This comment has been minimized.

Copy link
Contributor Author

@vinicyusmacedo vinicyusmacedo commented Mar 3, 2019

@cjerdonek I believe that's the same issue (sorry for duplicating it).

@cjerdonek

This comment has been minimized.

Copy link
Member

@cjerdonek cjerdonek commented Mar 3, 2019

@vinicyusmacedo No problem, that's okay. Just to clarify, does this mean your PR will completely resolve issue #6097? We can just make a note on that issue.

@vinicyusmacedo

This comment has been minimized.

Copy link
Contributor Author

@vinicyusmacedo vinicyusmacedo commented Mar 3, 2019

@cjerdonek it will, since it splits the @ from the requirement line and treats it as any other URL requirement.

I forgot to mention, but I had issues with setuptools too because of this bug, pretty much as in #6097 and #6306

@javabrett

This comment has been minimized.

Copy link
Contributor

@javabrett javabrett commented Jun 9, 2019

What is the status here - is it an accepted bug and looking-for-help, or work-underway/under-review, or issue-likely-rejected/won't-fix? This is biting me so just wondering whether it is time to pitch-in.

@vinicyusmacedo

This comment has been minimized.

Copy link
Contributor Author

@vinicyusmacedo vinicyusmacedo commented Jun 9, 2019

@javabrett it is currently under review. #6203
Ping @cjerdonek @uranusjr

@lock lock bot added the S: auto-locked label Oct 16, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
5 participants
You can’t perform that action at this time.