--extra-index-url does not work with pipenv install

[manihamidi]

Is the only way to install package on private index (specifically one which has public subdependencies on PyPI) to "manually" fill the "Pipfile" and do pipenv install as implied here?

In other words, pipenv install pkgname --extra-index-url <link-to-index> doesn't work (neither does pipenv install pkgname -i <link-to-index> (as it shouldn't since -i corresponds to --index-url which is not desirable in this context.)

Describe your environment

1. OS Type: windows 7
2. Python version: $ python -V 2.7
3. Pipenv version: $ pipenv --version 9.0.3

Expected result

I expect for package to be installed from the private index, and its sub-dependencies (specified in its setup.py) be installed from the public index (PyPI). Ideally the additional [[sources]] paragraph would be added to the Pipefile and the private package's "index" key would point to this "extra-index-url" as its source.

I understand that this service just may not be provided by pipenv (yet?), but I wonder if there is some existing solution that i'm not finding.

Actual result

pipenv can't find the private package to collect/install it. This is using either the -i or --extra-index-url as mentioned above.

Error:  An error occurred while installing pkgname!
  Could not find a version that satisfies the requirement pkgname (from versions: )
No matching distribution found for pkgname

Steps to replicate

• Create package called pkgname.
• Use install_requires in its setup.py to specify some public package as a dependency (e.g., pandas)
• Upload it to private index.
• Try to install it with pipenv

pipenv install pkgname --extra-index-url <link-to-index>

[mmoya]

In my case, pipenv explicitedly complains that Private repos not supported. Skipping.

$ pipenv --version
pipenv, version 9.0.3

$ pipenv install private-package '--extra-index-url https://xxxx:xxxx@pypi.internal/simple/'
Adding private-package to Pipfile's [packages]…
Installing --index-url https://pypi.internal/simple/…
/usr/lib/python3.6/site-packages/pipenv/vendor/requirements/parser.py:44: UserWarning: Private repos not supported. Skipping.
  warnings.warn('Private repos not supported. Skipping.')
…

So requirements/parser.py is vendored from pip.

It turned out that pypa/pip#3728 was recently merged. It allows variable expansion in requirements file, so you can have now, in pip, something like --extra-index-url=https://$PRIVATE_PYPI_USER:$PRIVATE_PYPI_PASS@pypi.internal/simple/ and everything will work as expected as long as you have those vars set in environment.

Maybe combining this new feature (by updating vendor) and using a custom package index will solve once and for all the problem with private repos. See for example #856.

For me the $PIP_EXTRA_INDEX_URL mentioned there works with pipenv install foo but doesn't work with pipenv lock.

[kennethreitz]

You need to specify the index in your pipfile:

https://docs.pipenv.org/advanced/#specifying-package-indexes

[AndreasPresthammer]

What do you do if your extra index require credentials?

[kennethreitz]

in the next version, you can use environment variables

[AndreasPresthammer]

I got it working with running pipenv install --skip-lock. Is that a reasonable workaround until the next version comes? Any eta on that version btw?

[kennethreitz]

probably within the next few days

[CodePint]

Yeah this is incredibly frustrating, there needs to be a way to specify an extra index URL using env vars.

We use pip-compile due to its speed and simplicity... Now a third party tool is insisting our compiled requirements.txt is recompiled with pipenv.

Fine whatever I can deal with that...except we can't because we don't have a Pipfile and rely on a private pypi for internal packages.

What would you suggest we do here?
Can we build a Pipfile at run time and populate it from the locked requirements.txt? ...lot of work even if it possible.

[merwok]

There are tools to convert, for example https://dephell.readthedocs.io/cmd-deps-convert.html

But generally it’s best to push back against tools or services that force one specific packaging tool and ask them to support standards.

[matteius]

@CodePint You just commented on an issue from 2018 without much context so its hard to give you a reasonably answer. We actually don't pass extra index URLs anymore and favor index restricted packages by default. You can pass --index to specify which index you want the package to be restricted to.

[CodePint]

There are tools to convert, for example https://dephell.readthedocs.io/cmd-deps-convert.html

But generally it’s best to push back against tools or services that force one specific packaging tool and ask them to support standards

This looks like it could be promising, thank you very much, I had not come across this.

Unfortunately the situation is what it is for now, but I will certainly try to raise the issue with thd tool maintainer.

[CodePint]

@CodePint You just commented on an issue from 2018 without much context so its hard to give you a reasonably answer. We actually don't pass extra index URLs anymore and favor index restricted packages by default. You can pass --index to specify which index you want the package to be restricted to.

I appreciate that, I should have raised a new issue in hindsight. Several hours later and knowing more about how --extra-index-url functions, I can see the flaws with it's implementation in pip.

I will have another look at the suggested approach with something like devpi. Thanks for taking the time to reply.