Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Un-deprecate --process-dependency-links until an alternative is implemented #4187

Closed
multimeric opened this issue Dec 19, 2016 · 72 comments
Closed

Comments

@multimeric
Copy link

@multimeric multimeric commented Dec 19, 2016

  • Pip version: 8.1.1
  • Python version: 3.5.2
  • Operating System: Ubuntu 16.04

Description:

The --process-dependency-links was re-added to pip a while ago because there are a number of valid use-cases for the flag: #1955

For this reason, it should probably be un-deprecated until an implementation of PEP 508/440 is in pip. The dependency_links argument to setuptools's setup is also still documented and not deprecated: http://setuptools.readthedocs.io/en/latest/setuptools.html#dependencies-that-aren-t-in-pypi

What I've run:

$ pip install request --process-dependency-links
Collecting request
  Downloading request-0.0.12.tar.gz
  DEPRECATION: Dependency Links processing has been deprecated and will be removed in a future release.
... 
@Kxrr
Copy link

@Kxrr Kxrr commented May 11, 2017

Agreed. Current behaviour of pip for dependency-links really sucks.
#4295
#3610

Loading

@jleclanche
Copy link

@jleclanche jleclanche commented Sep 19, 2017

[Wrote in #3939, moving my comment here]

So, here's the main issue: If I don't want to use a requirements.txt file (because, say, I want declarative dependencies all specified in setup.cfg), how am I supposed to specify a URL as dependency?

This doesn't work:

[options]
install_requires = 
  requests==2.18.4
  https://github.com/example/example_lib/archive/master.zip

I also think dependency links are weird and fine to drop, but canonically how is this use case served if not with those?

Loading

@jleclanche
Copy link

@jleclanche jleclanche commented Sep 19, 2017

Another issue is that dependency_links apparently cannot be specified in setup.cfg, only as setup.py arguments. If they are to be undeprecated, that should be fixed.

Loading

@robertour
Copy link

@robertour robertour commented Sep 20, 2017

Is there any news regarding this? Is there any alternative in progress? dependency_links_ seems to be the only option for internal/private distribution of libraries.

Instead of deprecating it, it should be fixed, a lot of people forget or get confuse because they forget to put both the name-version of the library in the the install_requires and the dependency link that contains the tarbal together with the egg. As an alternative, I would love to see that @jleclanche has suggested, a simple link to the tarball in the install_requires (specifying the egg could be optional).

I don't think we are alone: https://stackoverflow.com/questions/3472430/how-can-i-make-setuptools-install-a-package-thats-not-on-pypi

Loading

@RonnyPfannschmidt
Copy link
Contributor

@RonnyPfannschmidt RonnyPfannschmidt commented Sep 20, 2017

@robertour tools like devpi support a more clear approach since years now

Loading

@RonnyPfannschmidt
Copy link
Contributor

@RonnyPfannschmidt RonnyPfannschmidt commented Sep 20, 2017

@dstufft btw - i would like to argue that the existene of devpi demonstrates a more clean solution to the problem making dependency links unnecessary

IMHO index inheritance + white listing/blacklisting is more consistent and more comprehensible at a ops level

Loading

@robertour
Copy link

@robertour robertour commented Sep 20, 2017

@RonnyPfannschmidt, thanks for the suggestion, it seems I am missing something important (discussions about this deprecation have been going for too long) but generally speaking I cannot agree more with this comment from reddit:

devpi could be a solution, but bringing another service into the game that needs maintenance and having to wait for some administrator to setup, will not get me into working very soon. Also we still would have the need to build packages and then put them onto devip (or has devpi a feature to track git repositories??)

Ideally, I would like an alternative that won't take me more than adding a file in my library, or few lines in the setup.py.

I basically don't understand the underlying decision of removing it, I can see how it can lead to bad practices and insecurities, but as long as it is keep for internal use, it should not be a major issue.

Loading

@pradyunsg
Copy link
Member

@pradyunsg pradyunsg commented Oct 2, 2017

#4175 means that PEP 508 URL support will be there in pip 10. I think this can be closed.

Thoughts @pfmoore @dstufft?

Loading

@pfmoore
Copy link
Member

@pfmoore pfmoore commented Oct 2, 2017

I thought 86b0779 meant that the @ syntax cannot be used for dependencies, so this means #4175 isn't a solution for this issue. The comments over there were that we shouldn't implement @ support for dependencies until PyPI was able to block uploads that used it (because we don't want installing something from PyPI to be able to grab stuff from arbitrary URLs, presumably).

Loading

@rgerkin
Copy link

@rgerkin rgerkin commented Oct 13, 2017

It shouldn't be deprecated until there is an alternative in place, i.e. PEP 508 support. Until then, too many people are using it.

Loading

@ghost
Copy link

@ghost ghost commented Nov 21, 2017

What's the proper workflow for this?

Let 's say there's a library that I need to add a feature to. I fork it on Github. Feature isn't going to be merged any time soon, so I have a tool that I set to use my Github fork instead of PyPi.

Now all of my users have to add --process-dependency-links when installing my tool with pip, which is a deprecated and even once removed flag?

Is there some option in setup.py that I'm missing or is there really no way around this? It seems like the only viable way to do this is to push up a forked pypi package. That is just going to add to user's confusion once your pull request does get merged, anyway.

Loading

@pradyunsg
Copy link
Member

@pradyunsg pradyunsg commented May 17, 2018

Let's come around to this.

--process-dependency-links does not really have an alternative today. So, I'll suggest that we go ahead and un-deprecate it.

However, the flag does mean that pip would reach out to arbitrary URLs -- it should have a warning attached to it about this.

Loading

@ziima
Copy link

@ziima ziima commented Jan 29, 2019

Hi guys, I just like to note that the deprecation pathway in this case was way too short. I know that dependency links have been marked as deprecated for quite some time, but PEP 508 URLs, which can be used to replace them, haven't been implemented until 18.1. As a result there was only a 3 month long window to switch from dependency links to URL requirements, which is very short time for large projects.

Loading

newskylabs added a commit to newskylabs/newskylabs-temple that referenced this issue Mar 6, 2019
deKlerk referenced this issue in deKlerk/ghpythonremote Mar 6, 2019
Replaced pip.main() call with subprocess.check_call()
Solved installation issue reported in pilcru#12
dojeda added a commit to quetz-al/quetzal-client that referenced this issue Mar 21, 2019
Found a suitable solution to dependency_link hell using @ at the dependency
declaration as suggested in:
pypa/pip#4187 (comment)
@KevinMars
Copy link

@KevinMars KevinMars commented May 9, 2019

@rgerkin Hi, I am trying to follow your instructions to no avail,

Searching for PACKAGE@ git+ssh://git@bitbucket.org:OWNER/PACKAGE.git@BRANCH
Reading https://pypi.org/simple/PACKAGE/
Couldn't find index page for 'PACKAGE' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/

PACKAGE@ git+ssh://git@bitbucket.org:OWNER/PACKAGE.git@BRANCH, this is in install_requires.

Would you have an idea why I am getting the above?

Loading

@rgerkin
Copy link

@rgerkin rgerkin commented May 9, 2019

@KevinMars There are a few differences between my example and what you have, including the use of git_ssh, bitbucket, a.git suffix, a named branch, and no version tag. Maybe one or more of those things is leading pip to search on PyPI instead of at your URL. What version of pip are you using?

Loading

@osjerick
Copy link

@osjerick osjerick commented May 17, 2019

To remark something that I've found: Using setup.py to install the package with python setup.py install still requires declarations of external dependencies in dependency_links.

In your setup.py file:

import os, sys
from setuptools import setup, find_packages

def read_requirements():
    """Parse requirements from requirements.txt."""
    reqs_path = os.path.join('.', 'requirements.txt')
    with open(reqs_path, 'r') as f:
        requirements = [line.rstrip() for line in f]
    return requirements

setup(
    ..., # Other stuff here
    install_requires=read_requirements(),
    )

Loading

@jw-cpnet
Copy link

@jw-cpnet jw-cpnet commented Jun 13, 2019

@rgerkin Thanks for sharing this solution. But what if I'm using pbr to setup my Python package? How to adapt this to fit pbr?

Loading

@justinsantoro
Copy link

@justinsantoro justinsantoro commented Jun 20, 2019

@KevinMars I have the same exact issue. Did you ever figure out the fix? I am trying to require a specific branch of a private bitbucket repo over SSH.

Loading

@PhilippeFerreiraDeSousa
Copy link

@PhilippeFerreiraDeSousa PhilippeFerreiraDeSousa commented Jul 1, 2019

I just realized --process-dependency-links didn't existed anymore. I'm grateful for all the work the community does. Trying to justify the decision in never ending discussions and a maze of issue closing and redirection was the chosen solution but I still think leaving this option would have harmed nobody.

Loading

@lock lock bot added the S: auto-locked label Jul 31, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.