Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipenv install --target /path/to/my/virtualenv/folder #746

Closed
foxmask opened this issue Sep 28, 2017 · 46 comments
Closed

pipenv install --target /path/to/my/virtualenv/folder #746

foxmask opened this issue Sep 28, 2017 · 46 comments

Comments

@foxmask
Copy link

@foxmask foxmask commented Sep 28, 2017

Hi,

Could it be possible to create the virtualenv in the folder of our choice instead of ~/.local/share/virtualenvs/ ?

for example

pipenv install --target ~/fox_venv/projectA

which will create the venv in ~/fox_venv/projectA and not in ~/.local/share/virtualenvs/projectA-XdFl1243x

Regards

@nateprewitt
Copy link
Member

@nateprewitt nateprewitt commented Sep 28, 2017

Hey @foxmask, you can set the PIPENV_VENV_IN_PROJECT environment variable to have pipenv place the .venv directory inside your project rather than with the others in .local.

This is likely the closest we'll get to your current request for now. Hopefully that helps accomplish what you're looking for. Thanks for taking the time to check in on this!

@foxmask
Copy link
Author

@foxmask foxmask commented Sep 28, 2017

I saw it but the ~/fox_venv/projectA/.venv/ folder is not the folder I've chosen, but if it's how that has to work, I bow.

@davipo
Copy link

@davipo davipo commented Sep 28, 2017

See https://docs.pipenv.org/advanced.html#custom-virtual-environment-location
I've got this line in ~/.bash_profile (on macOS):
export WORKON_HOME=$HOME/.virtualenvs
Maybe you could set WORKON_HOME when switching projects.

@hellupline
Copy link

@hellupline hellupline commented Oct 6, 2017

the pip install --target have other uses than using a venv

I use pip install --target dist/ libs/lib1 libs/lib2 .... app/src to create a dist.tar.gz with all dependencies and then I use docker add plus PYTHONPATH to bundle the application with custom libraries

I would be really good to use pipenv install --target dist/ with its lockfile

@rajasimon
Copy link

@rajasimon rajasimon commented Nov 30, 2017

I want this feature too. I've the private repo to be included in vendor/ folder. It will be nice pipenv have this feature like

botter-library = {git = "ssh://git@server/user/repo", ref = "v0.0.1", target = 'vendor'}
@mdeguzis
Copy link

@mdeguzis mdeguzis commented Feb 2, 2018

It is a shame that there can't be a hook that just sets WORKON_HOME for that invocation or set's it for the project context.

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Feb 2, 2018

@mdeguzis the workaround we've suggested in the past is to use a tool like direnv to handle the context switching for you

@jakul
Copy link

@jakul jakul commented Mar 5, 2018

I was able to workaround with:

pipenv run pip install --target dist/
@kevinkjt2000
Copy link

@kevinkjt2000 kevinkjt2000 commented May 11, 2018

It seems that supporting this feature would not be too much work, since pipenv can already do this:

pipenv run pip install -r <(pipenv lock -r) --target dist/
@uranusjr
Copy link
Member

@uranusjr uranusjr commented May 12, 2018

@kevinkjt2000 Yeah, our code base is so easy you never contributed anything.

@kevinkjt2000
Copy link

@kevinkjt2000 kevinkjt2000 commented May 12, 2018

@uranusjr I acknowledge that you have a collaborator badge, but that is a rather sarcastic comment to make... I encourage you to encourage potential new-comers instead of steering the github comment section into frivolous, insulting conversations. If you would like more contributors; instead, try confirming or denying comments as they are made. Sarcasm is rather off-putting. I used the word seems, because of how unsure I was.

My comment pointed out that pipenv is capable of knowing what requirements are to be installed with lock -r and that pip is capable of installing that list of requirements. In my mind, pip seems to be the foundation behind how pipenv installs packages; however, as I am new to pipenv it only seems that way. I thought that in making the comment on an issue where it was stated that This is likely the closest we'll get to your current request for now. on late September last year, that perhaps someone more knowledgeable would re-visit and consider if it was easier after a few months of contributions and given that bash shell one-liner that I posted.

@kevinkjt2000
Copy link

@kevinkjt2000 kevinkjt2000 commented May 12, 2018

That being said, I'm willing to take a crack at this. I shall poke around for a bit, and see if there is a place to inject the --target option to pip, or perhaps I shall find out that pip is not the backbone of this project that I thought it was. Who knows?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 12, 2018

@kevinkjt2000 @uranusjr is as much of a core maintainer as it is possible to be at the moment, no matter what his tag says, but you're right that you are just trying to help and I'm quite sure he misunderstood your intentions, so firstly apologies for that

Technically you're right, pipenv can really support anything because you can pipenv run <x> -- but I understand the need for this (we have somewhat complicated infrastructure and tooling built around vendoring libraries ourselves so I do feel your pain. However, I think we want to first get our core library 'well oiled', then focus on these things.

@kevinkjt2000
Copy link

@kevinkjt2000 kevinkjt2000 commented May 12, 2018

However, I think we want to first get our core library 'well oiled', then focus on these things.
pipenv can really support anything because you can pipenv run <x>

This? https://github.com/pypa/pipenv/blob/master/pipenv/core.py

I noticed this call to pipenv run in do_create_virtualenv:

delegator.run('pipenv run pewtwo toggleglobalsitepackages')

Is the goal to not allow more calls to pipenv run in this core.py file (since it is being "well oiled")? I think this is the location where --target could be supported, something like "pipenv run pip install --target {0} -r {1}" (where {1} would be some temporary file that contains pipenv lock -r's output)?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 12, 2018

@kevinkjt2000 no, i don't mean literally 'the core.py file in pipenv', I mean we will consider adding functionality like --target when the rest of the project works the way we want it to. Also, that's not quite how the call would work, since that's a global virtualenv instruction and --target is simply an instruction of where to dump the files that would normally go into the python site-packages folder. Toggling global site packages just says 'put that in the system site packages folder' or 'put it in the virtualenv site packages folder', not 'put it specifically right here'.

@kevinkjt2000
Copy link

@kevinkjt2000 kevinkjt2000 commented May 12, 2018

Whoops, I should have said this is the file instead of this is the location.

@AlJohri
Copy link
Contributor

@AlJohri AlJohri commented Aug 28, 2018

if you can't use -r <(pipenv lock -r) (because you're using sh), try:

pipenv lock -r | pipenv run pip install -r /dev/stdin

Update: I use the following command for my lambda:

pipenv lock -r | sed 's/-e //g' | pipenv run pip install --upgrade -r /dev/stdin --target build
@HerrSpace
Copy link

@HerrSpace HerrSpace commented Oct 15, 2018

I would also like this feature to be able to deploy whole virtualenvs. Since virtualenvs can not easily be moved they must be created at their final location.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Oct 16, 2018

While I do not object to this feature per se, shipping whole virtual environments is never a good idea. I would recommend looking into alternative packaging methods such as zipapp.

@HerrSpace
Copy link

@HerrSpace HerrSpace commented Oct 16, 2018

Since you are featuring PEP 441 I want to point people reading towards https://github.com/pantsbuild/pex too. It's makes use of PEP 441 and is pretty straight forward.

While that is all nice PEP 441 has some downsides: It doesn't allow bundling a specific python interpreter, django settings.pys get harder to handle, etc. Why do you think it's never a good idea?

@rgarrigue
Copy link

@rgarrigue rgarrigue commented Oct 16, 2018

Someone recommended me cf_Freeze a while ago https://anthony-tuininga.github.io/cx_Freeze/ to package whole python setup. Any opinion on this tool ?

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Oct 16, 2018

@HerrSpace Because it is against the developers’ intentions. pypa/virtualenv#1035

cx_Freeze is a fine tool, but has a quite specific purpose. It does not support all use cases as a result, and is probably not @HerrSpace is looking for (considering the mention of Django).

@HerrSpace
Copy link

@HerrSpace HerrSpace commented Oct 16, 2018

@uranusjr Interesting, thank you!

That said I have previously deployed prebuild virtualenvs with some success in very homogeneous systems. The CI and production systems were basically identical.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Oct 17, 2018

Yeah, I wouldn’t be surprised it works if the base systems are more or less the same. This is essentially how many package managers (e.g. APT) works, really, by pre-building binaries with a well-controlled environment, and copy them to the exact location that works. Python ecosystems are unfortunately not that well-controlled, and don’t generally incline to support this usage. It would work, but you’ll be on your own.

@HerrSpace
Copy link

@HerrSpace HerrSpace commented Oct 18, 2018

Aye, I guess PEP 441 is where the journey is going.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Oct 18, 2018

Oh and—if you’re into the bleeding edge stuff, PEP 582 might be worth a read. It is essentially node_modules for Python.

@johneast
Copy link

@johneast johneast commented Oct 18, 2018

We would like this feature.

We have a large team working on a project. We would like to create the virtual environment in the root of the project. We wouldn't check the environment in, but we would check the IDE config in which would reduce the number of steps team members need to take in order to get up and running on the project.

@bigunyak
Copy link

@bigunyak bigunyak commented Nov 20, 2018

There is a number of people including myself requesting this feature, so maybe the ticket should be reopened?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Nov 20, 2018

The feature exists. You can set WORKON_HOME to use collision free hashes of the filesystem path and have those stored in the specified folder, or you can set PIPENV_VENV_IN_PROJECT to generate the virtualenv locally in your project root, of you can simply create a .venv file locally with a path to the actual environment.

Note that a few people asking for a feature isn’t a justification for including it or an explanation for why the need isn’t met by existing functionality. We have a process if you do feel there is a real need after evaluating existing options; you will need to submit a pull request to the peeps directory with a markdown document explaining the feature and why it is needed, what features exist, etc. basically demonstrating that you have given due consideration to the API change you are asking for. The reason for this is that we can’t really take a feature back once it’s been included, so we include things carefully.

@Jakobovski
Copy link

@Jakobovski Jakobovski commented Apr 23, 2019

Justification: pipenv's default location is in my home dir. At my office the home dir is over network mount and rather slow both in terms of latency and throughput. Thus lunching a python program that has lots of imports can take up to 20 seconds! If instead I install the venv on a local mount then load time drops to 2 seconds.

The problem with PIPENV_VENV_IN_PROJECT is that sometimes I have multiple copies of the same git repository and it is annoying to maintain multiple pipenvs. It is often easier to maintain multiple copies than stashing or committing partial work.

@jpcope
Copy link

@jpcope jpcope commented May 7, 2019

Justification: In a world with docker containers and developers that prefer to use a "local environment" to develop rather than a docker container because things like interactive debugging, live-reload, and IDE integrations are more easily connected without extra work; I require a feature like this so that I can select a path that does not collide with the developer location ( set via the PIPENV_VENV_IN_PROJECT env variable in the project dir ) with the location ultimately used in build utils and in the released container.

I find I keep looking for a solution and come to one conclusion consistently: do not use pipenv to maintain the virtualenv. This is the pattern I see in the comments and I hope others do as well.

I offer this feedback because I respect your efforts and want to ensure a portion of your user's needs are well understood. I am not asking for feature creep as much as I am asking for support for an edge case we seem to find ourselves within.

@pirate
Copy link

@pirate pirate commented Nov 11, 2019

Here's a concrete example of what @jpcope is describing, which is impossible with the current version of pipenv. I believe this encapsulates the essence of this issue/feature request and shows a strong use case that demonstrates why --target is needed.

WORKDIR /app
COPY ./Pipfile /app/
COPY ./Pipfile.lock /app/
RUN env PIPENV_SKIP_LOCK=1 \
        PIPENV_VENV_IN_PROJECT=1 \
        PIPENV_IGNORE_VIRTUALENVS=1 \
        pipenv install --clear --skip-lock --target=/app/.docker-venv  # --target allows us to specify a different venv path so as not to conflict with .venv
ENV PATH="/app/.docker-venv/bin:${PATH}"
...

This way the entire /app code directory can be mounted as a docker volume shared with the host, without the host's /app/.venvconflicting with the venv inside the container ( /app/.docker-venv).
They need to be separate venv directories entirely because the host might be a different OS with incompatibly built pip packages, e.g. macOS host .venv, alpine container .docker-venv.

@hoshsadiq
Copy link

@hoshsadiq hoshsadiq commented Dec 30, 2019

Any chance we can have this reopened and have this functionality tracked as a feature request?

In addition, to all of this, it would be nice to be able to specify the target of the Pipfile so that we can simply do Pipenv install without having to think about anything else.

@AlJohri
Copy link
Contributor

@AlJohri AlJohri commented Dec 30, 2019

You can use the PIPENV_PIPFILE environment variable for your latter request: https://pipenv-fork.readthedocs.io/en/latest/advanced.html#configuration-with-environment-variables

@hoshsadiq
Copy link

@hoshsadiq hoshsadiq commented Dec 30, 2019

Sorry, I think what I wrote wasn't clear enough, I'm not looking for a custom Pipfile location. I was referring to being able to specify any such target option as described in this thread within the Pipfile, as suggested by @rajasimon, or better yet within a global section:

# either
[global]
target = vendor
# or
[packages]
requests = {version = "*", target = 'vendor'}

Ideally the former option

@hoshsadiq
Copy link

@hoshsadiq hoshsadiq commented Dec 30, 2019

Is raising a PEEP the appropriate method of raising a feature request as described in this issue?

@shamoons
Copy link

@shamoons shamoons commented Mar 23, 2020

I was able to workaround with:

pipenv run pip install --target dist/

But then this won't add to the Pipfile, right?

@KA6WKE
Copy link

@KA6WKE KA6WKE commented Apr 3, 2020

This worked for me on Windows 10, guessing it would work on other OSes.

export WORKON_HOME=c:\\path\\to\\virtualenvs

Successfully created virtual environment!
Virtualenv location: c:\Users\xxxxxxxx\django_backend-4wqc7qLw
Launching subshell in virtual environment… 
$ which python
/c/Users/xxxxxxxx/django_backend-4wqc7qLw/Scripts/python
@madhums
Copy link

@madhums madhums commented Apr 8, 2020

As @pirate highlights ⬆️ --target is needed in order to make it work with docker. Very much agree!

Is there any possibility (in the current version) to specify where the venv folder should be?

@madhums
Copy link

@madhums madhums commented Apr 8, 2020

@nateprewitt (and the pipenv community) can you please reconsider adding a --target option?

@maquino1985
Copy link

@maquino1985 maquino1985 commented Apr 15, 2020

This is required to work in real world docker based deployment scenarios. Please re-open this.

@dalefrancum
Copy link

@dalefrancum dalefrancum commented Apr 20, 2020

Update: I use the following command for my lambda:

pipenv lock -r | sed 's/-e //g' | pipenv run pip install --upgrade -r /dev/stdin --target build

Thanks @AlJohri , this worked like a charm.

@Ordiel
Copy link

@Ordiel Ordiel commented Jun 17, 2020

For those trying to get it to work with Docker, you can set PIPENV_VENV_IN_PROJECT then mount a dir on your host to project_path/.venv.

With respect to where does the files actually land, you can always rely on the good ol' symlynks right?

Using those 2 I managed to set up a branch specific caching mechanism for my Flask project dependencies build by Jenkins multibranch within a Docker container, so I know it works in case the tip helps anybody

@samfrances
Copy link

@samfrances samfrances commented Feb 16, 2021

This would be a very welcome feature.

@Jawmo
Copy link

@Jawmo Jawmo commented Apr 30, 2021

Just found this as well when looking for a solution and worked like a charm:

"Just create an empty .venv directory yourself. Pipenv will pick this up automatically."

https://stackoverflow.com/questions/57919110/how-to-set-pipenv-venv-in-project-on-per-project-basis

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet