Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for environment variables in requirement specifiers #3516

Closed
gioele opened this issue Feb 7, 2019 · 6 comments · Fixed by #4533
Closed

Support for environment variables in requirement specifiers #3516

gioele opened this issue Feb 7, 2019 · 6 comments · Fixed by #4533

Comments

@gioele
Copy link

gioele commented Feb 7, 2019

Pipenv should accept bash-like enviroment variables in requirement specifiers and substitute them on the fly when pipev is run.

Is your feature request related to a problem? Please describe.

One possible use of this feature is installing packages from private GitHub repositories via OAuth. Right now this is not possible without storing the OAuth token in plain text in Pipenv like, for example:

[packages]
private-package = {editable = true,
    git = "https://1234abcdef56789:x-oauth-basic@github.com/myorg/private-package.git",
    ref = "v2.1"}

Describe the solution you'd like

It would be nice if bash-like enviroment variables where accepted and substituted on the fly when pipenv is run. For example:

pipenv install 'git+https://${GIT_TOKEN}:x-oauth-basic@github.com/myorg/private-package.git@v2.1#egg=private-package'

(Please note the single quotes.)

Describe alternatives you've considered

In plain pip it is possible to use pip install -r <(envsubst < requirements.txt). This solution, however, is not possible with pipenv.

Additional context

See also pypa/pip#6244.

@slavaatsig
Copy link

Yes! Yes! Please!
I was about to open this issue but found that it’s already created!
For an example for Google App Engine projects that are relying on private repositories, I can use my OS environment variables to safely install dependencies in a local environment and deploy to GAE using pip but cannot do the same using Pipenv without hard-coding credentials in the Pipfile.

@slavaatsig
Copy link

Also, I believe that PEEP-002 should be updated accordingly. Implementation of this feature really helps from security point of view.

In order to keep using Pipenv but avoiding to specify credentials in Pipfile (that is tracked in the VCS), I have implemented some scripts in Pipfile that are replacing custom placeholders with content from environment variables before installation and then do the opposite upon installation.

Describe alternatives you've considered

Having pip-like support of the environment variables in Pipfile would be much beter than these akward wheel on a cratched work-arounds.

@manskx
Copy link

manskx commented Apr 25, 2019

Yes Please !

@uhurusurfa
Copy link

uhurusurfa commented Dec 1, 2019

Happy to help implement if there is buy in to provide a solution.... need this

@skyoo2003
Copy link

skyoo2003 commented Apr 23, 2020

I add a comment because I thought this problem was not being handled.

Pip supports installation by passing the environment variables (
but, only in limited format) to the VCS URL, but it seems strange that Pipenv does not support them. ( https://pip.pypa.io/en/stable/reference/pip_install/#id10 )

I am just waiting for this issue to be resolved. Please!

@roger-domenech-aguilera

It will be very helpful. Thanks!

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this issue Jan 21, 2022
2022.1.8 (2022-01-08)
=====================

Bug Fixes
---------

- Remove the extra parentheses around the venv prompt.  `#4877 <https://github.com/pypa/pipenv/issues/4877>`_
- Fix a bug of installation fails when extra index url is given.  `#4881 <https://github.com/pypa/pipenv/issues/4881>`_
- Fix regression where lockfiles would only include the hashes for releases for the platform generating the lockfile  `#4885 <https://github.com/pypa/pipenv/issues/4885>`_
- Fix the index parsing to reject illegal requirements.txt.  `#4899 <https://github.com/pypa/pipenv/issues/4899>`_

2021.11.23 (2021-11-23)
=======================

Bug Fixes
---------

- Update ``charset-normalizer`` from ``2.0.3`` to ``2.0.7``, this fixes an import error on Python 3.6.  `#4865 <https://github.com/pypa/pipenv/issues/4865>`_
- Fix a bug of deleting a virtualenv that is not managed by Pipenv.  `#4867 <https://github.com/pypa/pipenv/issues/4867>`_
- Fix a bug that source is not added to ``Pipfile`` when index url is given with ``pipenv install``.  `#4873 <https://github.com/pypa/pipenv/issues/4873>`_

2021.11.15 (2021-11-15)
=======================

Bug Fixes
---------

- Return an empty dict when ``PIPENV_DONT_LOAD_ENV`` is set.  `#4851 <https://github.com/pypa/pipenv/issues/4851>`_
- Don't use ``sys.executable`` when inside an activated venv.  `#4852 <https://github.com/pypa/pipenv/issues/4852>`_

Vendored Libraries
------------------

- Drop the vendored ``jinja2`` dependency as it is not needed any more.  `#4858 <https://github.com/pypa/pipenv/issues/4858>`_
- Update ``click`` from ``8.0.1`` to ``8.0.3``, to fix a problem with bash completion.  `#4860 <https://github.com/pypa/pipenv/issues/4860>`_
- Drop unused vendor ``chardet``.  `#4862 <https://github.com/pypa/pipenv/issues/4862>`_

Improved Documentation
----------------------

- Fix the documentation to reflect the fact that special characters must be percent-encoded in the URL.  `#4856 <https://github.com/pypa/pipenv/issues/4856>`_

2021.11.9 (2021-11-09)
======================

Features & Improvements
-----------------------

- Replace ``click-completion`` with ``click``'s own completion implementation.  `#4786 <https://github.com/pypa/pipenv/issues/4786>`_

Bug Fixes
---------

- Fix a bug that ``pipenv run`` doesn't set environment variables correctly.  `#4831 <https://github.com/pypa/pipenv/issues/4831>`_
- Fix a bug that certifi can't be loaded within ``notpip``'s vendor library. This makes several objects of ``pip`` fail to be imported.  `#4833 <https://github.com/pypa/pipenv/issues/4833>`_
- Fix a bug that ``3.10.0`` can be found be python finder.  `#4837 <https://github.com/pypa/pipenv/issues/4837>`_

Vendored Libraries
------------------

- Update ``pythonfinder`` from ``1.2.8`` to ``1.2.9``.  `#4837 <https://github.com/pypa/pipenv/issues/4837>`_

2021.11.5.post0 (2021-11-05)
============================

Bug Fixes
---------

- Fix a regression that ``pipenv shell`` fails to start a subshell.  `#4828 <https://github.com/pypa/pipenv/issues/4828>`_
- Fix a regression that ``pip_shims`` object isn't imported correctly.  `#4829 <https://github.com/pypa/pipenv/issues/4829>`_

2021.11.5 (2021-11-05)
======================

Features & Improvements
-----------------------

- Avoid sharing states but create project objects on demand. So that most integration test cases are able to switch to a in-process execution method.  `#4757 <https://github.com/pypa/pipenv/issues/4757>`_
- Shell-quote ``pip`` commands when logging.  `#4760 <https://github.com/pypa/pipenv/issues/4760>`_

Bug Fixes
---------

- Ignore empty .venv in rood dir and create project name base virtual environment  `#4790 <https://github.com/pypa/pipenv/issues/4790>`_

Vendored Libraries
------------------

- Update vendored dependencies
  - ``attrs`` from ``20.3.0`` to ``21.2.0``
  - ``cerberus`` from ``1.3.2`` to ``1.3.4``
  - ``certifi`` from ``2020.11.8`` to ``2021.5.30``
  - ``chardet`` from ``3.0.4`` to ``4.0.0``
  - ``click`` from ``7.1.2`` to ``8.0.1``
  - ``distlib`` from ``0.3.1`` to ``0.3.2``
  - ``idna`` from ``2.10`` to ``3.2``
  - ``importlib-metadata`` from ``2.0.0`` to ``4.6.1``
  - ``importlib-resources`` from ``3.3.0`` to ``5.2.0``
  - ``jinja2`` from ``2.11.2`` to ``3.0.1``
  - ``markupsafe`` from ``1.1.1`` to ``2.0.1``
  - ``more-itertools`` from ``5.0.0`` to ``8.8.0``
  - ``packaging`` from ``20.8`` to ``21.0``
  - ``pep517`` from ``0.9.1`` to ``0.11.0``
  - ``pipdeptree`` from ``1.0.0`` to ``2.0.0``
  - ``ptyprocess`` from ``0.6.0`` to ``0.7.0``
  - ``python-dateutil`` from ``2.8.1`` to ``2.8.2``
  - ``python-dotenv`` from ``0.15.0`` to ``0.19.0``
  - ``pythonfinder`` from ``1.2.5`` to ``1.2.8``
  - ``requests`` from ``2.25.0`` to ``2.26.0``
  - ``shellingham`` from ``1.3.2`` to ``1.4.0``
  - ``six`` from ``1.15.0`` to ``1.16.0``
  - ``tomlkit`` from ``0.7.0`` to ``0.7.2``
  - ``urllib3`` from ``1.26.1`` to ``1.26.6``
  - ``zipp`` from ``1.2.0`` to ``3.5.0``

  Add new vendored dependencies
  - ``charset-normalizer 2.0.3``
  - ``termcolor 1.1.0``
  - ``tomli 1.1.0``
  - ``wheel 0.36.2``  `#4747 <https://github.com/pypa/pipenv/issues/4747>`_
- Drop the dependencies for Python 2.7 compatibility purpose.  `#4751 <https://github.com/pypa/pipenv/issues/4751>`_
- Switch the dependency resolver from ``pip-tools`` to `pip`.

  Update vendor libraries:
  - Update ``requirementslib`` from ``1.5.16`` to ``1.6.1``
  - Update ``pip-shims`` from ``0.5.6`` to ``0.6.0``
  - New vendor ``platformdirs 2.4.0``  `#4759 <https://github.com/pypa/pipenv/issues/4759>`_

Improved Documentation
----------------------

- remove prefixes on install commands for easy copy/pasting  `#4792 <https://github.com/pypa/pipenv/issues/4792>`_
- Officially drop support for Python 2.7 and Python 3.5.  `#4261 <https://github.com/pypa/pipenv/issues/4261>`_

2021.5.29 (2021-05-29)
======================

Bug Fixes
---------

- Fix a bug where passing --skip-lock when PIPFILE has no [SOURCE] section throws the error: "tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'"  `#4141 <https://github.com/pypa/pipenv/issues/4141>`_
- Fix bug where environment wouldn't activate in paths containing & and $ symbols  `#4538 <https://github.com/pypa/pipenv/issues/4538>`_
- Fix a bug that ``importlib-metadata`` from the project's dependencies conflicts with that from ``pipenv``'s.  `#4549 <https://github.com/pypa/pipenv/issues/4549>`_
- Fix a bug where ``pep508checker.py`` did not expect double-digit Python minor versions (e.g. "3.10").  `#4602 <https://github.com/pypa/pipenv/issues/4602>`_
- Fix bug where environment wouldn't activate in paths containing () and [] symbols  `#4615 <https://github.com/pypa/pipenv/issues/4615>`_
- Fix bug preventing use of pipenv lock --pre  `#4642 <https://github.com/pypa/pipenv/issues/4642>`_

Vendored Libraries
------------------

- Update ``packaging`` from ``20.4`` to ``20.8``.  `#4591 <https://github.com/pypa/pipenv/issues/4591>`_

2020.11.15 (2020-11-15)
=======================

Features & Improvements
-----------------------

- Support expanding environment variables in requirement URLs.  `#3516 <https://github.com/pypa/pipenv/issues/3516>`_
- Show warning message when a dependency is skipped in locking due to the mismatch of its markers.  `#4346 <https://github.com/pypa/pipenv/issues/4346>`_

Bug Fixes
---------

- Fix a bug that executable scripts with leading backslash can't be executed via ``pipenv run``.  `#4368 <https://github.com/pypa/pipenv/issues/4368>`_
- Fix a bug that VCS dependencies always satisfy even if the ref has changed.  `#4387 <https://github.com/pypa/pipenv/issues/4387>`_
- Restrict the acceptable hash type to SHA256 only.  `#4517 <https://github.com/pypa/pipenv/issues/4517>`_
- Fix the output of ``pipenv scripts`` under Windows platform.  `#4523 <https://github.com/pypa/pipenv/issues/4523>`_
- Fix a bug that the resolver takes wrong section to validate constraints.  `#4527 <https://github.com/pypa/pipenv/issues/4527>`_

Vendored Libraries
------------------

- Update vendored dependencies:
    - ``colorama`` from ``0.4.3`` to ``0.4.4``
    - ``python-dotenv`` from ``0.10.3`` to ``0.15.0``
    - ``first`` from ``2.0.1`` to ``2.0.2``
    - ``iso8601`` from ``0.1.12`` to ``0.1.13``
    - ``parse`` from ``1.15.0`` to ``1.18.0``
    - ``pipdeptree`` from ``0.13.2`` to ``1.0.0``
    - ``requests`` from ``2.23.0`` to ``2.25.0``
    - ``idna`` from ``2.9`` to ``2.10``
    - ``urllib3`` from ``1.25.9`` to ``1.26.1``
    - ``certifi`` from ``2020.4.5.1`` to ``2020.11.8``
    - ``requirementslib`` from ``1.5.15`` to ``1.5.16``
    - ``attrs`` from ``19.3.0`` to ``20.3.0``
    - ``distlib`` from ``0.3.0`` to ``0.3.1``
    - ``packaging`` from ``20.3`` to ``20.4``
    - ``six`` from ``1.14.0`` to ``1.15.0``
    - ``semver`` from ``2.9.0`` to ``2.13.0``
    - ``toml`` from ``0.10.1`` to ``0.10.2``
    - ``cached-property`` from ``1.5.1`` to ``1.5.2``
    - ``yaspin`` from ``0.14.3`` to ``1.2.0``
    - ``resolvelib`` from ``0.3.0`` to ``0.5.2``
    - ``pep517`` from ``0.8.2`` to ``0.9.1``
    - ``zipp`` from ``0.6.0`` to ``1.2.0``
    - ``importlib-metadata`` from ``1.6.0`` to ``2.0.0``
    - ``importlib-resources`` from ``1.5.0`` to ``3.3.0``  `#4533 <https://github.com/pypa/pipenv/issues/4533>`_

Improved Documentation
----------------------

- Fix suggested pyenv setup to avoid using shimmed interpreter  `#4534 <https://github.com/pypa/pipenv/issues/4534>`_

2020.11.4 (2020-11-04)
======================

Features & Improvements
-----------------------

- Add a new command ``pipenv scripts`` to display shortcuts from Pipfile.  `#3686 <https://github.com/pypa/pipenv/issues/3686>`_
- Retrieve package file hash from URL to accelerate the locking process.  `#3827 <https://github.com/pypa/pipenv/issues/3827>`_
- Add the missing ``--system`` option to ``pipenv sync``.  `#4441 <https://github.com/pypa/pipenv/issues/4441>`_
- Add a new option pair ``--header/--no-header`` to ``pipenv lock`` command,
  which adds a header to the generated requirements.txt  `#4443 <https://github.com/pypa/pipenv/issues/4443>`_

Bug Fixes
---------

- Fix a bug that percent encoded characters will be unquoted incorrectly in the file URL.  `#4089 <https://github.com/pypa/pipenv/issues/4089>`_
- Fix a bug where setting PIPENV_PYTHON to file path breaks environment name  `#4225 <https://github.com/pypa/pipenv/issues/4225>`_
- Fix a bug that paths are not normalized before comparison.  `#4330 <https://github.com/pypa/pipenv/issues/4330>`_
- Handle Python major and minor versions correctly in Pipfile creation.  `#4379 <https://github.com/pypa/pipenv/issues/4379>`_
- Fix a bug that non-wheel file requirements can be resolved successfully.  `#4386 <https://github.com/pypa/pipenv/issues/4386>`_
- Fix a bug that ``pexept.exceptions.TIMEOUT`` is not caught correctly because of the wrong import path.  `#4424 <https://github.com/pypa/pipenv/issues/4424>`_
- Fix a bug that compound TOML table is not parsed correctly.  `#4433 <https://github.com/pypa/pipenv/issues/4433>`_
- Fix a bug that invalid Python paths from Windows registry break ``pipenv install``.  `#4436 <https://github.com/pypa/pipenv/issues/4436>`_
- Fix a bug that function calls in ``setup.py`` can't be parsed rightly.  `#4446 <https://github.com/pypa/pipenv/issues/4446>`_
- Fix a bug that dist-info inside ``venv`` directory will be mistaken as the editable package's metadata.  `#4480 <https://github.com/pypa/pipenv/issues/4480>`_
- Make the order of hashes in resolution result stable.  `#4513 <https://github.com/pypa/pipenv/issues/4513>`_

Vendored Libraries
------------------

- Update ``tomlkit`` from ``0.5.11`` to ``0.7.0``.  `#4433 <https://github.com/pypa/pipenv/issues/4433>`_
- Update ``requirementslib`` from ``1.5.13`` to ``1.5.14``.  `#4480 <https://github.com/pypa/pipenv/issues/4480>`_

Improved Documentation
----------------------

- Discourage homebrew installation in installation guides.  `#4013 <https://github.com/pypa/pipenv/issues/4013>`_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants