Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create PEEP-006: extra pip arguments #3859

Closed
wants to merge 1 commit into from
Closed

Create PEEP-006: extra pip arguments #3859

wants to merge 1 commit into from

Conversation

con-f-use
Copy link

@con-f-use con-f-use commented Jul 18, 2019

See


PEEP-006: Pass extra pip arguments on a per-entry basis

This PEEP proposes the introduction of an argument extra_pip_options in Pipfile
package entries, which is intended to pass additional options to the invocation of pip.

Background

Some packages do not install without passing additional arguments to pip,
resulting in failure to lock or install with Pipenv.

As of writing this is the case for most packages that use a pyproject.toml
file with setuptools in editable mode.
These require pip to be invoked with --no-use-pep518.
While it was proposed to solve this by having Pipenv retry with this option,
as soon as the specfic error message (editable mode is not supported for pyproject.toml-style projects [...] you may pass --no-use-pep517 [...]) is encountered,
the author proposes the more general solution outlined in the proposal section.
Pip's ability to use environment variables for the names of its option is of no use here.
This is because the environment variable NO_USE_PEP518 would be applied to all pip invocations,
not just the one package, which setup.py cannot run correctly without the option.

A less esoteric package for which PEEP-006 would be helpful is pycurl.
Here is an excerpt from pycurl's setup.py:

    def configure_windows(self):
        # Windows users have to pass --curl-dir parameter to specify path
        # to libcurl, because there is no curl-config on windows at all.
        curl_dir = scan_argv(self.argv, "--curl-dir=")
        if curl_dir is None:
            fail("Please specify --curl-dir=/path/to/built/libcurl")

The lack of --curl-dir does not results in a failure to lock,
since Pipenv cannot retrive the necessary metadata on Windows systems.
This is owned to a failirue running setup.py.
Installation is done via wheel and succeeds.

Implementation of PEEP-006 could provide this mandatory option --curl-dir
for pycurl, when used in conjunctions with the platform_system == 'Windows' marker:

[packages]
pycurl = {version = "*", platform_system = "== 'Windows'", extra_pip_options=['--curl-dir=/path/to/curl/config']}
pycurl = {version = "*", platform_system = "!= 'Windows'"}

Additionally this woudl give the user fine control over pip to leverage such
options as:

$ pip install --help

[...]

  --proxy <proxy>             Specify a proxy in the form
                              [user:passwd@]proxy.server:port.
  --retries <retries>         Maximum number of retries each connection should
                              attempt (default 5 times).
  --timeout <sec>             Set the socket timeout (default 15 seconds).

Proposal

Introduce an extra keyword to a package entry in a Pipfile called extra_pip_options.
This keyword shall be of a list type containing pip command line options and their values.

A Pipfile using the new keyword would contain lines similar to these:

[packages]
somepack = {version = "*", extra_pip_options=['--no-use-pep518', '--timeout=60', '--retries=20']}

The list items would be passed to any invocation of pip just like command
line arguments:

pip --no-use-pep518 --timeout=60 --retries=20 ...

Should Pipenv already pass a given argument, the user should get a warning
and the argument Pipenv would use without extra_pip_options should be overwritten,
i.e. the one in extra_pip_options should be used over its value without extra_pip_options.

For some packages it might be useful to support expansion of environment variables:

[packages]
pycurl = {version = "*", platform_system = "== 'Windows'", extra_pip_options=['--curl-dir=${CURL_CONFIG_PATH}']}

Where CURL_CONFIG_PATH would be the variable name of such an environment variable.


Author: con-f-use@gmx.net

@con-f-use con-f-use changed the title Create PEEP-006.md Create PEEP-006: extra pip arguments Jul 18, 2019
@frostming
Copy link
Contributor

IMHO this looks a bit awkward and may break the reproducibility of the Pipfile. For example, the option --curl-dir=${CURL_CONFIG_PATH} may not be usable on the production server. For some globally applied options, do we have to duplicate them on each package?

I know the concern and problem you are seeking to solve behind this, but maybe we could have a better solution.

@con-f-use
Copy link
Author

I'm open to different solutions, that's just the best I could come up with. You are welcome to present one. The way I see it, reproducibility is already broken for these cases.

@frostming frostming added the Status: Requires Approval This issue requires additional approval to move forward. label Aug 2, 2019
@oz123 oz123 added the rotten label Apr 16, 2022
@matteius
Copy link
Member

matteius commented Sep 5, 2022

@con-f-use by chance -- we just merged in a PR that does basically this: #5283

It will be in the next release.

@matteius matteius closed this Sep 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rotten Status: Requires Approval This issue requires additional approval to move forward.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants