Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipenv install --index <my-pypi-registry> <my-package> generates a Pipfile that pipenv lock can't handle #4873

Closed
ben-g- opened this issue Nov 17, 2021 · 5 comments · Fixed by #4874

Comments

@ben-g-
Copy link

ben-g- commented Nov 17, 2021

I start from a completely blank slate -- no Pipfile, no Pipfile.lock, no virtual environment. And I execute a command like

pipenv install --index <my-pypi-registry> <my-package>

where <my-pypi-registry> is a private PyPI registry (not a mirror of the public PyPI registry!) that hosts the package <my-package>.

The result of the command is:

  • a virtual environment is successfully created
  • and a Pipfile is generated
  • but locking fails.
...
Successfully created virtual environment!
Virtualenv location: C:\tmp\tmp-venv\.venv
Creating a Pipfile for this project...
Installing <my-package>...
Adding <my-package> to Pipfile's [packages]...
Installation Succeeded
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
 Locking...Building requirements...
Resolving dependencies...
Locking Failed!

CRITICAL:pipenv.patched.notpip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement <my-package> (from versions: none)
[ResolutionFailure]:   File "c:\python391\lib\site-packages\pipenv\resolver.py", line 743, in _main
[ResolutionFailure]:       resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
[ResolutionFailure]:   File "c:\python391\lib\site-packages\pipenv\resolver.py", line 704, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "c:\python391\lib\site-packages\pipenv\resolver.py", line 685, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "c:\python391\lib\site-packages\pipenv\utils.py", line 1370, in resolve_deps
[ResolutionFailure]:       results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
[ResolutionFailure]:   File "c:\python391\lib\site-packages\pipenv\utils.py", line 1099, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "c:\python391\lib\site-packages\pipenv\utils.py", line 877, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: No matching distribution found for <my-package>

The generated Pipfile looks like this:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
<my-package> = {version = "*", index = "<my-pypi-registry>"}
...

That looks reasonable, but pipenv lock fails on it with the error shown above.

However, if I manually change the Pipfile to look like this, then pipenv lock succeeds, even though this seems essentially the same:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "<my-pypi-registry>"
verify_ssl = true
name = "mypypi"

[packages]
<my-package> = {version = "*", index = "mypypi"}
...

If (again starting from a completely blank slate) I replace --index with --extra-index-url, so my command looks like this:

pipenv install --extra-index-url <my-pypi-registry> <my-package>

then the command's output looks the same, meaning

  • a virtual environment is successfully created
  • and a Pipfile is generated
  • but locking fails with the same error as above.

However, in this case, the generated Pipfile is clearly wrong, in that it doesn't reference <my-pypi-registry> at all, so it's not surprising that pipenv lock fails on it. It looks like this:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
<my-package> = "*"
...

In summary, this might be two separate bugs.

  1. pipenv lock fails on the seemingly correct Pipfile generated by pipenv install --index ....
  2. pipenv install --extra-index-url ... generates an incorrect Pipfile.

I'm using pipenv version 2021.11.15, which I installed using pip.

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this issue Jan 21, 2022
2022.1.8 (2022-01-08)
=====================

Bug Fixes
---------

- Remove the extra parentheses around the venv prompt.  `#4877 <https://github.com/pypa/pipenv/issues/4877>`_
- Fix a bug of installation fails when extra index url is given.  `#4881 <https://github.com/pypa/pipenv/issues/4881>`_
- Fix regression where lockfiles would only include the hashes for releases for the platform generating the lockfile  `#4885 <https://github.com/pypa/pipenv/issues/4885>`_
- Fix the index parsing to reject illegal requirements.txt.  `#4899 <https://github.com/pypa/pipenv/issues/4899>`_

2021.11.23 (2021-11-23)
=======================

Bug Fixes
---------

- Update ``charset-normalizer`` from ``2.0.3`` to ``2.0.7``, this fixes an import error on Python 3.6.  `#4865 <https://github.com/pypa/pipenv/issues/4865>`_
- Fix a bug of deleting a virtualenv that is not managed by Pipenv.  `#4867 <https://github.com/pypa/pipenv/issues/4867>`_
- Fix a bug that source is not added to ``Pipfile`` when index url is given with ``pipenv install``.  `#4873 <https://github.com/pypa/pipenv/issues/4873>`_

2021.11.15 (2021-11-15)
=======================

Bug Fixes
---------

- Return an empty dict when ``PIPENV_DONT_LOAD_ENV`` is set.  `#4851 <https://github.com/pypa/pipenv/issues/4851>`_
- Don't use ``sys.executable`` when inside an activated venv.  `#4852 <https://github.com/pypa/pipenv/issues/4852>`_

Vendored Libraries
------------------

- Drop the vendored ``jinja2`` dependency as it is not needed any more.  `#4858 <https://github.com/pypa/pipenv/issues/4858>`_
- Update ``click`` from ``8.0.1`` to ``8.0.3``, to fix a problem with bash completion.  `#4860 <https://github.com/pypa/pipenv/issues/4860>`_
- Drop unused vendor ``chardet``.  `#4862 <https://github.com/pypa/pipenv/issues/4862>`_

Improved Documentation
----------------------

- Fix the documentation to reflect the fact that special characters must be percent-encoded in the URL.  `#4856 <https://github.com/pypa/pipenv/issues/4856>`_

2021.11.9 (2021-11-09)
======================

Features & Improvements
-----------------------

- Replace ``click-completion`` with ``click``'s own completion implementation.  `#4786 <https://github.com/pypa/pipenv/issues/4786>`_

Bug Fixes
---------

- Fix a bug that ``pipenv run`` doesn't set environment variables correctly.  `#4831 <https://github.com/pypa/pipenv/issues/4831>`_
- Fix a bug that certifi can't be loaded within ``notpip``'s vendor library. This makes several objects of ``pip`` fail to be imported.  `#4833 <https://github.com/pypa/pipenv/issues/4833>`_
- Fix a bug that ``3.10.0`` can be found be python finder.  `#4837 <https://github.com/pypa/pipenv/issues/4837>`_

Vendored Libraries
------------------

- Update ``pythonfinder`` from ``1.2.8`` to ``1.2.9``.  `#4837 <https://github.com/pypa/pipenv/issues/4837>`_

2021.11.5.post0 (2021-11-05)
============================

Bug Fixes
---------

- Fix a regression that ``pipenv shell`` fails to start a subshell.  `#4828 <https://github.com/pypa/pipenv/issues/4828>`_
- Fix a regression that ``pip_shims`` object isn't imported correctly.  `#4829 <https://github.com/pypa/pipenv/issues/4829>`_

2021.11.5 (2021-11-05)
======================

Features & Improvements
-----------------------

- Avoid sharing states but create project objects on demand. So that most integration test cases are able to switch to a in-process execution method.  `#4757 <https://github.com/pypa/pipenv/issues/4757>`_
- Shell-quote ``pip`` commands when logging.  `#4760 <https://github.com/pypa/pipenv/issues/4760>`_

Bug Fixes
---------

- Ignore empty .venv in rood dir and create project name base virtual environment  `#4790 <https://github.com/pypa/pipenv/issues/4790>`_

Vendored Libraries
------------------

- Update vendored dependencies
  - ``attrs`` from ``20.3.0`` to ``21.2.0``
  - ``cerberus`` from ``1.3.2`` to ``1.3.4``
  - ``certifi`` from ``2020.11.8`` to ``2021.5.30``
  - ``chardet`` from ``3.0.4`` to ``4.0.0``
  - ``click`` from ``7.1.2`` to ``8.0.1``
  - ``distlib`` from ``0.3.1`` to ``0.3.2``
  - ``idna`` from ``2.10`` to ``3.2``
  - ``importlib-metadata`` from ``2.0.0`` to ``4.6.1``
  - ``importlib-resources`` from ``3.3.0`` to ``5.2.0``
  - ``jinja2`` from ``2.11.2`` to ``3.0.1``
  - ``markupsafe`` from ``1.1.1`` to ``2.0.1``
  - ``more-itertools`` from ``5.0.0`` to ``8.8.0``
  - ``packaging`` from ``20.8`` to ``21.0``
  - ``pep517`` from ``0.9.1`` to ``0.11.0``
  - ``pipdeptree`` from ``1.0.0`` to ``2.0.0``
  - ``ptyprocess`` from ``0.6.0`` to ``0.7.0``
  - ``python-dateutil`` from ``2.8.1`` to ``2.8.2``
  - ``python-dotenv`` from ``0.15.0`` to ``0.19.0``
  - ``pythonfinder`` from ``1.2.5`` to ``1.2.8``
  - ``requests`` from ``2.25.0`` to ``2.26.0``
  - ``shellingham`` from ``1.3.2`` to ``1.4.0``
  - ``six`` from ``1.15.0`` to ``1.16.0``
  - ``tomlkit`` from ``0.7.0`` to ``0.7.2``
  - ``urllib3`` from ``1.26.1`` to ``1.26.6``
  - ``zipp`` from ``1.2.0`` to ``3.5.0``

  Add new vendored dependencies
  - ``charset-normalizer 2.0.3``
  - ``termcolor 1.1.0``
  - ``tomli 1.1.0``
  - ``wheel 0.36.2``  `#4747 <https://github.com/pypa/pipenv/issues/4747>`_
- Drop the dependencies for Python 2.7 compatibility purpose.  `#4751 <https://github.com/pypa/pipenv/issues/4751>`_
- Switch the dependency resolver from ``pip-tools`` to `pip`.

  Update vendor libraries:
  - Update ``requirementslib`` from ``1.5.16`` to ``1.6.1``
  - Update ``pip-shims`` from ``0.5.6`` to ``0.6.0``
  - New vendor ``platformdirs 2.4.0``  `#4759 <https://github.com/pypa/pipenv/issues/4759>`_

Improved Documentation
----------------------

- remove prefixes on install commands for easy copy/pasting  `#4792 <https://github.com/pypa/pipenv/issues/4792>`_
- Officially drop support for Python 2.7 and Python 3.5.  `#4261 <https://github.com/pypa/pipenv/issues/4261>`_

2021.5.29 (2021-05-29)
======================

Bug Fixes
---------

- Fix a bug where passing --skip-lock when PIPFILE has no [SOURCE] section throws the error: "tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'"  `#4141 <https://github.com/pypa/pipenv/issues/4141>`_
- Fix bug where environment wouldn't activate in paths containing & and $ symbols  `#4538 <https://github.com/pypa/pipenv/issues/4538>`_
- Fix a bug that ``importlib-metadata`` from the project's dependencies conflicts with that from ``pipenv``'s.  `#4549 <https://github.com/pypa/pipenv/issues/4549>`_
- Fix a bug where ``pep508checker.py`` did not expect double-digit Python minor versions (e.g. "3.10").  `#4602 <https://github.com/pypa/pipenv/issues/4602>`_
- Fix bug where environment wouldn't activate in paths containing () and [] symbols  `#4615 <https://github.com/pypa/pipenv/issues/4615>`_
- Fix bug preventing use of pipenv lock --pre  `#4642 <https://github.com/pypa/pipenv/issues/4642>`_

Vendored Libraries
------------------

- Update ``packaging`` from ``20.4`` to ``20.8``.  `#4591 <https://github.com/pypa/pipenv/issues/4591>`_

2020.11.15 (2020-11-15)
=======================

Features & Improvements
-----------------------

- Support expanding environment variables in requirement URLs.  `#3516 <https://github.com/pypa/pipenv/issues/3516>`_
- Show warning message when a dependency is skipped in locking due to the mismatch of its markers.  `#4346 <https://github.com/pypa/pipenv/issues/4346>`_

Bug Fixes
---------

- Fix a bug that executable scripts with leading backslash can't be executed via ``pipenv run``.  `#4368 <https://github.com/pypa/pipenv/issues/4368>`_
- Fix a bug that VCS dependencies always satisfy even if the ref has changed.  `#4387 <https://github.com/pypa/pipenv/issues/4387>`_
- Restrict the acceptable hash type to SHA256 only.  `#4517 <https://github.com/pypa/pipenv/issues/4517>`_
- Fix the output of ``pipenv scripts`` under Windows platform.  `#4523 <https://github.com/pypa/pipenv/issues/4523>`_
- Fix a bug that the resolver takes wrong section to validate constraints.  `#4527 <https://github.com/pypa/pipenv/issues/4527>`_

Vendored Libraries
------------------

- Update vendored dependencies:
    - ``colorama`` from ``0.4.3`` to ``0.4.4``
    - ``python-dotenv`` from ``0.10.3`` to ``0.15.0``
    - ``first`` from ``2.0.1`` to ``2.0.2``
    - ``iso8601`` from ``0.1.12`` to ``0.1.13``
    - ``parse`` from ``1.15.0`` to ``1.18.0``
    - ``pipdeptree`` from ``0.13.2`` to ``1.0.0``
    - ``requests`` from ``2.23.0`` to ``2.25.0``
    - ``idna`` from ``2.9`` to ``2.10``
    - ``urllib3`` from ``1.25.9`` to ``1.26.1``
    - ``certifi`` from ``2020.4.5.1`` to ``2020.11.8``
    - ``requirementslib`` from ``1.5.15`` to ``1.5.16``
    - ``attrs`` from ``19.3.0`` to ``20.3.0``
    - ``distlib`` from ``0.3.0`` to ``0.3.1``
    - ``packaging`` from ``20.3`` to ``20.4``
    - ``six`` from ``1.14.0`` to ``1.15.0``
    - ``semver`` from ``2.9.0`` to ``2.13.0``
    - ``toml`` from ``0.10.1`` to ``0.10.2``
    - ``cached-property`` from ``1.5.1`` to ``1.5.2``
    - ``yaspin`` from ``0.14.3`` to ``1.2.0``
    - ``resolvelib`` from ``0.3.0`` to ``0.5.2``
    - ``pep517`` from ``0.8.2`` to ``0.9.1``
    - ``zipp`` from ``0.6.0`` to ``1.2.0``
    - ``importlib-metadata`` from ``1.6.0`` to ``2.0.0``
    - ``importlib-resources`` from ``1.5.0`` to ``3.3.0``  `#4533 <https://github.com/pypa/pipenv/issues/4533>`_

Improved Documentation
----------------------

- Fix suggested pyenv setup to avoid using shimmed interpreter  `#4534 <https://github.com/pypa/pipenv/issues/4534>`_

2020.11.4 (2020-11-04)
======================

Features & Improvements
-----------------------

- Add a new command ``pipenv scripts`` to display shortcuts from Pipfile.  `#3686 <https://github.com/pypa/pipenv/issues/3686>`_
- Retrieve package file hash from URL to accelerate the locking process.  `#3827 <https://github.com/pypa/pipenv/issues/3827>`_
- Add the missing ``--system`` option to ``pipenv sync``.  `#4441 <https://github.com/pypa/pipenv/issues/4441>`_
- Add a new option pair ``--header/--no-header`` to ``pipenv lock`` command,
  which adds a header to the generated requirements.txt  `#4443 <https://github.com/pypa/pipenv/issues/4443>`_

Bug Fixes
---------

- Fix a bug that percent encoded characters will be unquoted incorrectly in the file URL.  `#4089 <https://github.com/pypa/pipenv/issues/4089>`_
- Fix a bug where setting PIPENV_PYTHON to file path breaks environment name  `#4225 <https://github.com/pypa/pipenv/issues/4225>`_
- Fix a bug that paths are not normalized before comparison.  `#4330 <https://github.com/pypa/pipenv/issues/4330>`_
- Handle Python major and minor versions correctly in Pipfile creation.  `#4379 <https://github.com/pypa/pipenv/issues/4379>`_
- Fix a bug that non-wheel file requirements can be resolved successfully.  `#4386 <https://github.com/pypa/pipenv/issues/4386>`_
- Fix a bug that ``pexept.exceptions.TIMEOUT`` is not caught correctly because of the wrong import path.  `#4424 <https://github.com/pypa/pipenv/issues/4424>`_
- Fix a bug that compound TOML table is not parsed correctly.  `#4433 <https://github.com/pypa/pipenv/issues/4433>`_
- Fix a bug that invalid Python paths from Windows registry break ``pipenv install``.  `#4436 <https://github.com/pypa/pipenv/issues/4436>`_
- Fix a bug that function calls in ``setup.py`` can't be parsed rightly.  `#4446 <https://github.com/pypa/pipenv/issues/4446>`_
- Fix a bug that dist-info inside ``venv`` directory will be mistaken as the editable package's metadata.  `#4480 <https://github.com/pypa/pipenv/issues/4480>`_
- Make the order of hashes in resolution result stable.  `#4513 <https://github.com/pypa/pipenv/issues/4513>`_

Vendored Libraries
------------------

- Update ``tomlkit`` from ``0.5.11`` to ``0.7.0``.  `#4433 <https://github.com/pypa/pipenv/issues/4433>`_
- Update ``requirementslib`` from ``1.5.13`` to ``1.5.14``.  `#4480 <https://github.com/pypa/pipenv/issues/4480>`_

Improved Documentation
----------------------

- Discourage homebrew installation in installation guides.  `#4013 <https://github.com/pypa/pipenv/issues/4013>`_
@Yuribtr
Copy link

Yuribtr commented Mar 28, 2022

Seems this problem resurrected in last release 2022.3.28 (at least).
I was unable to install python packages from my private repository. The same problem raised:

CRITICAL:pipenv.patched.notpip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement my-utils (from versions: none)

Pipfile is:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "http://nexus.localhost:8081/repository/pypi-internal/simple/"
verify_ssl = false
name = "nexus"

[dev-packages]
ipython = "*"
pytest = "*"
pytest-env = "*"
pytest-cov = "*"
flake8 = "*"
flake8-broken-line = "*"
flake8-builtins = "*"
flake8-commas = "*"
flake8-comprehensions = "*"
flake8-mutable = "*"
flake8-print = "*"
flake8-cognitive-complexity = "*"
flake8-functions = "*"
flake8-quotes = "*"
pre-commit = "*"
requests = "*"

[packages]
fastapi = "*"
uvicorn = "*"
gunicorn = "*"
python-json-logger = "*"
markdown2 = "*"
my-utils = {extras = ["fastapi"], version = "*",  index = "nexus"}

[requires]
python_version = "3.9"

[pipenv]
allow_prereleases = false

Also another problem raised, if in Pipfile I swap sources like this:

[[source]]
url = "http://nexus.localhost:8081/repository/pypi-internal/simple/"
verify_ssl = false
name = "nexus"

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

........

In above case I get:

CRITICAL:pipenv.patched.notpip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement flake8-comprehensions (from versions: none)

Solution:
pip install "pipenv==2022.1.8"

@matteius
Copy link
Member

@Yuribtr Well the example is going to be hard to reproduce since I don't have the private pypi package my-utils ... however I have a theory. Can you try removing the extras = ["fastapi"] and add that as its own dependency to your requirements? I suspect that could be causing the issue and neither the basics or advacned pipenv docs page talks about that syntax. If you find there is still an issue, could you open a new issues report for this?

@bkraft4257
Copy link

I have been able to reproduce this exact same problem. The only differences based upon the description are

  1. I am using a different private repository and a different package
  2. I did not use extras when specifying the package in the private repo. Just version and index. sdspytools = {version="==0.7.1", name="private-pypi"}

Installation of the dependencies went fine. I verified this by marking sure sdspytools was installed in the site-packages. Locking always fails. What I think is happening is that locking mechanism is only accessing the public repo. You can see the traceback message shows from versions: none. I used to see this error only when access to my private repository was not set up correctly.

I am using environment variables in my Pipfile to pass in the password and username to the URL. I thought this might be a problem so as a test I hard coded my username and password in the URL in the Pipfile. Unfortunately, the result was the same.

`
(cdi-data-blackdog) (my_python38) [ec2-user@ip-10-10-64-184 cdi-data-blackdog]$ pipenv install --dev

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.

Pipfile.lock (dc4640) out of date, updating to (300441)...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed!

CRITICAL:pipenv.patched.notpip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement sdspytools==0.7.1 (from versions: none)`

@bkraft4257
Copy link

Another piece of information that may be useful is when I run pipenv graph I can see that my package has been installed and there are no conflicts reported (at least none that I have noticed).

@matteius
Copy link
Member

matteius commented Apr 4, 2022

@bkraft4257 This is a closed issue you are commenting on, and also you did not indicate the version of pipenv you are running or the contents of an example Pipfile. If you would like support please ensure you are running the latest version, have checked recent opened issues and potentially file a new issue report. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants