Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing Python 3.9.8 with custom protected source fails #4856

Closed
mskrip opened this issue Nov 10, 2021 · 5 comments
Closed

Installing Python 3.9.8 with custom protected source fails #4856

mskrip opened this issue Nov 10, 2021 · 5 comments

Comments

@mskrip
Copy link

mskrip commented Nov 10, 2021

Issue description

When installing packages with pipenv with custom source protected by credentials fails with Python 3.9.8 (but works with 3.9.7).
Also tried using an older version of pipenv but it didn't work.

Expected result

The packages should install successfully.

Actual result

This is a snipped of logs of the failed installation:

[pipenv.exceptions.InstallError]: Looking in indexes: ***pypi-***.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/
[pipenv.exceptions.InstallError]: WARNING: 401 Error, Credentials not correct for https://pypi-***.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/attrs/
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement attrs==21.2.0 (from versions: none)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for attrs==21.2.0

Steps to replicate

  • Install Python 3.9.8 (default now for Github Actions Python version 3.9)
  • Configure custom source with credentials
  • run $ pipenv install

Not posting pipenv --support because it's the same as running on Github Actions on ubuntu-latest with Python 3.9, this might be the only relevant part of Pipfile:

[[source]]
name = "repo"
url = "https://aws:'${TOKEN}'@pypi-****.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/"
verify_ssl = true
@frostming
Copy link
Contributor

frostming commented Nov 10, 2021

Try this https://aws:${TOKEN}@....
Also the token should be percent-encoded.

@mskrip
Copy link
Author

mskrip commented Nov 10, 2021

The token is a JWT token created by aws codeartifact, so there are no percent-encodable characters. And without the apostrophes I get an ImportError: cannot import name 'project' from 'pipenv.core' .

But as I mentioned this works with Python 3.9.7

@frostming
Copy link
Contributor

frostming commented Nov 10, 2021

Can you provide the full error traceback? What version of Pipenv are you using?

@mskrip
Copy link
Author

mskrip commented Nov 10, 2021

It's the latest version of Pipenv 2021.11.09, here is a full traceback of replicating this in docker and trying to install requests

Installing requests...
Installing package: requests
Writing supplied requirement line to temporary file: 'requests'
Installing 'requests'
⠹ Installing requests...$ /root/.local/share/virtualenvs/root-BuDEOXnJ/bin/python -m pip install --pre --verbose --upgrade --exists-action=i -r /tmp/pipenv-nnx65vbj-requirements/pipenv-7eq0otob-requirement.txt -i 'https://aws:'"'"'<token>'"'"'@pypi-****.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/'
Using source directory: '/root/.local/share/virtualenvs/root-BuDEOXnJ/src'
Error:  An error occurred while installing requests!
Error text: Using pip 21.3.1 from /root/.local/share/virtualenvs/root-BuDEOXnJ/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://aws:****@pypi-****.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/

WARNING: 401 Error, Credentials not correct for https://pypi-****.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/requests/
ERROR: Could not find a version that satisfies the requirement requests (from versions: none)
ERROR: No matching distribution found for requests

Using pip 21.3.1 from /root/.local/share/virtualenvs/root-BuDEOXnJ/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://aws:****@pypi-****.d.codeartifact.eu-west-1.amazonaws.com/pypi/<repo>/simple/

✘ Installation Failed 

I can see the token is escaped very weirdly, so I tried your suggestion from above again (removing apostrophes) and after relocking the project it worked (but I can't get it to install on CI since I hit #4855).

So that's another workaround, but IMO it's still a bug since pipenv documentation suggests using the apostrophes which does not seem to work now :(.

@oz123
Copy link
Contributor

oz123 commented Nov 11, 2021

Seems to me that pipenv imports the wrong attrs.
It should have imported :

from pipenv.vendor import attrs

Used to be like this before the version bump. Here is the current one:

.

Here is the previous version:

from pipenv.vendor import attr

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this issue Jan 21, 2022
2022.1.8 (2022-01-08)
=====================

Bug Fixes
---------

- Remove the extra parentheses around the venv prompt.  `#4877 <https://github.com/pypa/pipenv/issues/4877>`_
- Fix a bug of installation fails when extra index url is given.  `#4881 <https://github.com/pypa/pipenv/issues/4881>`_
- Fix regression where lockfiles would only include the hashes for releases for the platform generating the lockfile  `#4885 <https://github.com/pypa/pipenv/issues/4885>`_
- Fix the index parsing to reject illegal requirements.txt.  `#4899 <https://github.com/pypa/pipenv/issues/4899>`_

2021.11.23 (2021-11-23)
=======================

Bug Fixes
---------

- Update ``charset-normalizer`` from ``2.0.3`` to ``2.0.7``, this fixes an import error on Python 3.6.  `#4865 <https://github.com/pypa/pipenv/issues/4865>`_
- Fix a bug of deleting a virtualenv that is not managed by Pipenv.  `#4867 <https://github.com/pypa/pipenv/issues/4867>`_
- Fix a bug that source is not added to ``Pipfile`` when index url is given with ``pipenv install``.  `#4873 <https://github.com/pypa/pipenv/issues/4873>`_

2021.11.15 (2021-11-15)
=======================

Bug Fixes
---------

- Return an empty dict when ``PIPENV_DONT_LOAD_ENV`` is set.  `#4851 <https://github.com/pypa/pipenv/issues/4851>`_
- Don't use ``sys.executable`` when inside an activated venv.  `#4852 <https://github.com/pypa/pipenv/issues/4852>`_

Vendored Libraries
------------------

- Drop the vendored ``jinja2`` dependency as it is not needed any more.  `#4858 <https://github.com/pypa/pipenv/issues/4858>`_
- Update ``click`` from ``8.0.1`` to ``8.0.3``, to fix a problem with bash completion.  `#4860 <https://github.com/pypa/pipenv/issues/4860>`_
- Drop unused vendor ``chardet``.  `#4862 <https://github.com/pypa/pipenv/issues/4862>`_

Improved Documentation
----------------------

- Fix the documentation to reflect the fact that special characters must be percent-encoded in the URL.  `#4856 <https://github.com/pypa/pipenv/issues/4856>`_

2021.11.9 (2021-11-09)
======================

Features & Improvements
-----------------------

- Replace ``click-completion`` with ``click``'s own completion implementation.  `#4786 <https://github.com/pypa/pipenv/issues/4786>`_

Bug Fixes
---------

- Fix a bug that ``pipenv run`` doesn't set environment variables correctly.  `#4831 <https://github.com/pypa/pipenv/issues/4831>`_
- Fix a bug that certifi can't be loaded within ``notpip``'s vendor library. This makes several objects of ``pip`` fail to be imported.  `#4833 <https://github.com/pypa/pipenv/issues/4833>`_
- Fix a bug that ``3.10.0`` can be found be python finder.  `#4837 <https://github.com/pypa/pipenv/issues/4837>`_

Vendored Libraries
------------------

- Update ``pythonfinder`` from ``1.2.8`` to ``1.2.9``.  `#4837 <https://github.com/pypa/pipenv/issues/4837>`_

2021.11.5.post0 (2021-11-05)
============================

Bug Fixes
---------

- Fix a regression that ``pipenv shell`` fails to start a subshell.  `#4828 <https://github.com/pypa/pipenv/issues/4828>`_
- Fix a regression that ``pip_shims`` object isn't imported correctly.  `#4829 <https://github.com/pypa/pipenv/issues/4829>`_

2021.11.5 (2021-11-05)
======================

Features & Improvements
-----------------------

- Avoid sharing states but create project objects on demand. So that most integration test cases are able to switch to a in-process execution method.  `#4757 <https://github.com/pypa/pipenv/issues/4757>`_
- Shell-quote ``pip`` commands when logging.  `#4760 <https://github.com/pypa/pipenv/issues/4760>`_

Bug Fixes
---------

- Ignore empty .venv in rood dir and create project name base virtual environment  `#4790 <https://github.com/pypa/pipenv/issues/4790>`_

Vendored Libraries
------------------

- Update vendored dependencies
  - ``attrs`` from ``20.3.0`` to ``21.2.0``
  - ``cerberus`` from ``1.3.2`` to ``1.3.4``
  - ``certifi`` from ``2020.11.8`` to ``2021.5.30``
  - ``chardet`` from ``3.0.4`` to ``4.0.0``
  - ``click`` from ``7.1.2`` to ``8.0.1``
  - ``distlib`` from ``0.3.1`` to ``0.3.2``
  - ``idna`` from ``2.10`` to ``3.2``
  - ``importlib-metadata`` from ``2.0.0`` to ``4.6.1``
  - ``importlib-resources`` from ``3.3.0`` to ``5.2.0``
  - ``jinja2`` from ``2.11.2`` to ``3.0.1``
  - ``markupsafe`` from ``1.1.1`` to ``2.0.1``
  - ``more-itertools`` from ``5.0.0`` to ``8.8.0``
  - ``packaging`` from ``20.8`` to ``21.0``
  - ``pep517`` from ``0.9.1`` to ``0.11.0``
  - ``pipdeptree`` from ``1.0.0`` to ``2.0.0``
  - ``ptyprocess`` from ``0.6.0`` to ``0.7.0``
  - ``python-dateutil`` from ``2.8.1`` to ``2.8.2``
  - ``python-dotenv`` from ``0.15.0`` to ``0.19.0``
  - ``pythonfinder`` from ``1.2.5`` to ``1.2.8``
  - ``requests`` from ``2.25.0`` to ``2.26.0``
  - ``shellingham`` from ``1.3.2`` to ``1.4.0``
  - ``six`` from ``1.15.0`` to ``1.16.0``
  - ``tomlkit`` from ``0.7.0`` to ``0.7.2``
  - ``urllib3`` from ``1.26.1`` to ``1.26.6``
  - ``zipp`` from ``1.2.0`` to ``3.5.0``

  Add new vendored dependencies
  - ``charset-normalizer 2.0.3``
  - ``termcolor 1.1.0``
  - ``tomli 1.1.0``
  - ``wheel 0.36.2``  `#4747 <https://github.com/pypa/pipenv/issues/4747>`_
- Drop the dependencies for Python 2.7 compatibility purpose.  `#4751 <https://github.com/pypa/pipenv/issues/4751>`_
- Switch the dependency resolver from ``pip-tools`` to `pip`.

  Update vendor libraries:
  - Update ``requirementslib`` from ``1.5.16`` to ``1.6.1``
  - Update ``pip-shims`` from ``0.5.6`` to ``0.6.0``
  - New vendor ``platformdirs 2.4.0``  `#4759 <https://github.com/pypa/pipenv/issues/4759>`_

Improved Documentation
----------------------

- remove prefixes on install commands for easy copy/pasting  `#4792 <https://github.com/pypa/pipenv/issues/4792>`_
- Officially drop support for Python 2.7 and Python 3.5.  `#4261 <https://github.com/pypa/pipenv/issues/4261>`_

2021.5.29 (2021-05-29)
======================

Bug Fixes
---------

- Fix a bug where passing --skip-lock when PIPFILE has no [SOURCE] section throws the error: "tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'"  `#4141 <https://github.com/pypa/pipenv/issues/4141>`_
- Fix bug where environment wouldn't activate in paths containing & and $ symbols  `#4538 <https://github.com/pypa/pipenv/issues/4538>`_
- Fix a bug that ``importlib-metadata`` from the project's dependencies conflicts with that from ``pipenv``'s.  `#4549 <https://github.com/pypa/pipenv/issues/4549>`_
- Fix a bug where ``pep508checker.py`` did not expect double-digit Python minor versions (e.g. "3.10").  `#4602 <https://github.com/pypa/pipenv/issues/4602>`_
- Fix bug where environment wouldn't activate in paths containing () and [] symbols  `#4615 <https://github.com/pypa/pipenv/issues/4615>`_
- Fix bug preventing use of pipenv lock --pre  `#4642 <https://github.com/pypa/pipenv/issues/4642>`_

Vendored Libraries
------------------

- Update ``packaging`` from ``20.4`` to ``20.8``.  `#4591 <https://github.com/pypa/pipenv/issues/4591>`_

2020.11.15 (2020-11-15)
=======================

Features & Improvements
-----------------------

- Support expanding environment variables in requirement URLs.  `#3516 <https://github.com/pypa/pipenv/issues/3516>`_
- Show warning message when a dependency is skipped in locking due to the mismatch of its markers.  `#4346 <https://github.com/pypa/pipenv/issues/4346>`_

Bug Fixes
---------

- Fix a bug that executable scripts with leading backslash can't be executed via ``pipenv run``.  `#4368 <https://github.com/pypa/pipenv/issues/4368>`_
- Fix a bug that VCS dependencies always satisfy even if the ref has changed.  `#4387 <https://github.com/pypa/pipenv/issues/4387>`_
- Restrict the acceptable hash type to SHA256 only.  `#4517 <https://github.com/pypa/pipenv/issues/4517>`_
- Fix the output of ``pipenv scripts`` under Windows platform.  `#4523 <https://github.com/pypa/pipenv/issues/4523>`_
- Fix a bug that the resolver takes wrong section to validate constraints.  `#4527 <https://github.com/pypa/pipenv/issues/4527>`_

Vendored Libraries
------------------

- Update vendored dependencies:
    - ``colorama`` from ``0.4.3`` to ``0.4.4``
    - ``python-dotenv`` from ``0.10.3`` to ``0.15.0``
    - ``first`` from ``2.0.1`` to ``2.0.2``
    - ``iso8601`` from ``0.1.12`` to ``0.1.13``
    - ``parse`` from ``1.15.0`` to ``1.18.0``
    - ``pipdeptree`` from ``0.13.2`` to ``1.0.0``
    - ``requests`` from ``2.23.0`` to ``2.25.0``
    - ``idna`` from ``2.9`` to ``2.10``
    - ``urllib3`` from ``1.25.9`` to ``1.26.1``
    - ``certifi`` from ``2020.4.5.1`` to ``2020.11.8``
    - ``requirementslib`` from ``1.5.15`` to ``1.5.16``
    - ``attrs`` from ``19.3.0`` to ``20.3.0``
    - ``distlib`` from ``0.3.0`` to ``0.3.1``
    - ``packaging`` from ``20.3`` to ``20.4``
    - ``six`` from ``1.14.0`` to ``1.15.0``
    - ``semver`` from ``2.9.0`` to ``2.13.0``
    - ``toml`` from ``0.10.1`` to ``0.10.2``
    - ``cached-property`` from ``1.5.1`` to ``1.5.2``
    - ``yaspin`` from ``0.14.3`` to ``1.2.0``
    - ``resolvelib`` from ``0.3.0`` to ``0.5.2``
    - ``pep517`` from ``0.8.2`` to ``0.9.1``
    - ``zipp`` from ``0.6.0`` to ``1.2.0``
    - ``importlib-metadata`` from ``1.6.0`` to ``2.0.0``
    - ``importlib-resources`` from ``1.5.0`` to ``3.3.0``  `#4533 <https://github.com/pypa/pipenv/issues/4533>`_

Improved Documentation
----------------------

- Fix suggested pyenv setup to avoid using shimmed interpreter  `#4534 <https://github.com/pypa/pipenv/issues/4534>`_

2020.11.4 (2020-11-04)
======================

Features & Improvements
-----------------------

- Add a new command ``pipenv scripts`` to display shortcuts from Pipfile.  `#3686 <https://github.com/pypa/pipenv/issues/3686>`_
- Retrieve package file hash from URL to accelerate the locking process.  `#3827 <https://github.com/pypa/pipenv/issues/3827>`_
- Add the missing ``--system`` option to ``pipenv sync``.  `#4441 <https://github.com/pypa/pipenv/issues/4441>`_
- Add a new option pair ``--header/--no-header`` to ``pipenv lock`` command,
  which adds a header to the generated requirements.txt  `#4443 <https://github.com/pypa/pipenv/issues/4443>`_

Bug Fixes
---------

- Fix a bug that percent encoded characters will be unquoted incorrectly in the file URL.  `#4089 <https://github.com/pypa/pipenv/issues/4089>`_
- Fix a bug where setting PIPENV_PYTHON to file path breaks environment name  `#4225 <https://github.com/pypa/pipenv/issues/4225>`_
- Fix a bug that paths are not normalized before comparison.  `#4330 <https://github.com/pypa/pipenv/issues/4330>`_
- Handle Python major and minor versions correctly in Pipfile creation.  `#4379 <https://github.com/pypa/pipenv/issues/4379>`_
- Fix a bug that non-wheel file requirements can be resolved successfully.  `#4386 <https://github.com/pypa/pipenv/issues/4386>`_
- Fix a bug that ``pexept.exceptions.TIMEOUT`` is not caught correctly because of the wrong import path.  `#4424 <https://github.com/pypa/pipenv/issues/4424>`_
- Fix a bug that compound TOML table is not parsed correctly.  `#4433 <https://github.com/pypa/pipenv/issues/4433>`_
- Fix a bug that invalid Python paths from Windows registry break ``pipenv install``.  `#4436 <https://github.com/pypa/pipenv/issues/4436>`_
- Fix a bug that function calls in ``setup.py`` can't be parsed rightly.  `#4446 <https://github.com/pypa/pipenv/issues/4446>`_
- Fix a bug that dist-info inside ``venv`` directory will be mistaken as the editable package's metadata.  `#4480 <https://github.com/pypa/pipenv/issues/4480>`_
- Make the order of hashes in resolution result stable.  `#4513 <https://github.com/pypa/pipenv/issues/4513>`_

Vendored Libraries
------------------

- Update ``tomlkit`` from ``0.5.11`` to ``0.7.0``.  `#4433 <https://github.com/pypa/pipenv/issues/4433>`_
- Update ``requirementslib`` from ``1.5.13`` to ``1.5.14``.  `#4480 <https://github.com/pypa/pipenv/issues/4480>`_

Improved Documentation
----------------------

- Discourage homebrew installation in installation guides.  `#4013 <https://github.com/pypa/pipenv/issues/4013>`_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants