Let user choose location of .venv

[woky]

It's irritating that you cannot specify custom virtual environment directory location. It makes pipenv unsuitable for production deployments. PIPENV_VENV_IN_PROJECT is suitable only for development environments.

Consider this scenario using docker-compose.yaml to set up a Docker container:

...

  install:
    image: python:3.8
    environment:
      PIPENV_VENV_IN_PROJECT: 1
    volumes:
      - /srv/deploy/app:/app:ro # Note that this is bind mounted read-only
      - /var/cache/libgen-tools-venv:/app/.venv
    # Pipenv writes by default so we pass --keep-outdated
    # because Pipfile.lock is on read-only filesystem.
    command: pipenv sync --keep-outdated

  periodic-batch-job-1:
    image: python:3.8
    environment:
      PIPENV_VENV_IN_PROJECT: 1
    volumes:
      - /srv/deploy/app:/app:ro # Note that this is bind mounted read-only
      - /var/cache/libgen-tools-venv:/app/.venv:ro
    command: pipenv run python3 /app/dostuff.py

  ...

Here the project is deployed out of Git/CI/CD into /srv/deploy/app and is mounted read-only to containers at /app, because more containers share the same code and they have no business overwriting it. But this means that /app/.venv cannot be created in containers.

This is very specific example I'm now just dealing with, but having virtual environment in /var/cache or any other volatile directory seems to be perfectly legal usage.

I've experimented with exporting VIRTUAL_ENV prior to running pipenv install but it is ignored.

This issue was previously closed here #746.

[techalchemy]

Hey there, sorry you're running into this frustration -- I've struggled with this myself and I definitely hear you. You have a couple of options already which address this use case broadly speaking -- not all of these will apply to your specific use case but I will include them in case they help anyone else:

1. Setting VIRTUAL_ENV should work ok, as should sourceing a virtual env before invoking any calls (though you'd have to do so ahead of each call);
2. You can include a .venv file which contains text with a path to a virtual env, e.g. /path/to/some/venv_base, which pipenv will then treat as the actual environment;
3. You can forego the virtualenv entirely and install using --system -- this would install directly into the container's environment obviously, so YMMV;
4. You can set WORKON_HOME to /var/cache/app_venvs and make sure to create the directory and give whomever will be writing the appropriate access; pipenv will then automatically create the virtualenv there based on a deterministic hash of the application path

As for the comment in there about pipenv writing everything it sees, yeah, that issue is also super annoying. I expect to tweak that API a bit sometime soon to try and make that a bit less painful.

Let me know if any of that can meet your use case -- thanks!

[rr326]

Perhaps this is the wrong thread for this comment, but I've been burned by this too. I was deploying my app to my production environment and it wasn't working. After a bunch of debugging, I remembered the PIPENV_VENV_IN_PROJECT issue, and how I had to set that in my workstation .zshenv, and that's not in my new environment. The way my mind works, I'd like my app environment to be self-contained. And setting PIPENV_PIPFILE should be enough. Instead, my cron job is PIPENV_VENV_IN_PROJECT=1 PIPENV_PIPFILE=${ROOT}/Pipfile pipenv run do_something.

It sounds like I could manually create a .venv file, which seems like it may be the right solution. But I would hope Pipenv would make these nuances go away.

2 cents.

[rr326]

Follow up after searching about the .venv file. No wonder I didn't know about it - it's not documented. And it doesn't help. Looking at #3134, is there a reason the initial logic couldn't be:

1. Given a root, if .venv file or dirctory, use that.
2. If not, if PIPENV_VENV_IN_PROJECT, create a local .venv
3. Etc.

It's kind of like git or many apps that walk up from the current directory, looking for config until you find it. So start at the most local level (the app root), check there, and if not, keep going.

[MegMM]

If PIPENV_VENV_IN_PROJECT is the ultimate solution for installing to a specific .venv location, can this be added to the documentation on https://pipenv.pypa.io/en/latest/ and https://pypi.org/project/pipenv/? Can the parameter be set in the pyvenv.cfg or activate.bat file? I personally use a thumb drive to work between different machines (due to network lag) and under different user names. My preferred solution would be a pipenv install --target type of solution (the most obvious). Otherwise a more documented method of using the parameter PIPENV_VENV_IN_PROJECT will do. Ultimately, I would need pipenv to install to my project's local .venv\ on the USB drive and not store anything to a specific user's account.

[uranusjr]

Documentation help is always welcomed! Feel free to submit a pull request and ping me for review.

[MegMM]

Hi! I would love to be able to contribute to docs, but am still mostly a confused newbie. I did try setting the PIPENV_VENV_IN_PROJECT in the .venv\pyvenv.cfg and that seems to work with the VS Code interpreter set to “..venv\Scripts\python.exe” at least in the state everything is in right now. I’m trying to run a VS Code/python 3.8/pipenv solution on a thumb drive for my coding project. Mostly got it working by making everything as vanilla possible and not setting any specific parameters, like the VS Code venv settings.

But here are a couple of documentation suggestions:

On page https://pipenv.pypa.io/en/latest/advanced/#configuration-with-environment-variables

PIPENV_VENV_IN_PROJECT¶
If set, creates .venv in your project directory.

I would modify it to read something to the effect

If set, creates and installs packages to the local .venv in your project directory”.

I think people look for “local” when they’re trying to figure this setup out. Also this would specify that it changes the installation target.

The other pertinent page https://pipenv.pypa.io/en/latest/install: In the subsection “Isolated Installation of Pipenv with Pipx” under “Installing Pipenv”? Is this what I should be doing instead of PIPENV_VENV_IN_PROJECT? Is pipx for developing a python app? Or for running/maintaining a working python app? Some differentiation between the 2 options might be worth it here.

Otherwise, it would be helpful to have a tiny mention on the “PIPENV_VENV_IN_PROJECT” parameter under the “Installing packages for your project” since it seems applicable here. Currently, the blurb is this

$ cd myproject
$ pipenv install requests
Pipenv will install the excellent Requests library and create a Pipfile for you in your project’s directory. The Pipfile is used to track which dependencies your project needs in case you need to re-install them, such as when you share your project with others. You should get output similar to this (although the exact paths shown will vary):

A possible tiny modification could be

Pipenv will install the excellent Requests library and create a Pipfile for you in your project’s directory. If you have set PIPENV_VENV_IN_PROJECT, Pipenv will install to your local .venv folder. The Pipfile is used to track which dependencies your project needs in case you need to re-install them, such as when you share your project with others. You should get output similar to this (although the exact paths shown will vary)

[OrangeDog]

Setting VIRTUAL_ENV should work ok,

To what? What does it do? It's not documented.
https://pipenv.pypa.io/en/latest/advanced/#configuration-with-environment-variables

I just want a simple option to give pipenv a path where it should create or use an existing virtal environment.

[ZelphirKaltstahl]

It is definitely possible to specify the location of the virtual env using a combination of the following:

• creating a virtual env using virtual env
• then using environment variables to configure pipenv
• then using pipenv to install packages into that virtual environment
• (then modifying PATH environment variable)

I will try to give an example later.

[OrangeDog]

@ZelphirKaltstahl but pipenv should create and activate the virtualenv as necessary, otherwise you way as well just use pip freeze.

[ZelphirKaltstahl]

@ZelphirKaltstahl but pipenv should create and activate the virtualenv as necessary, otherwise you way as well just use pip freeze.

That would be nice of course. There is a qualitative difference between pip freeze and a lock file such as Pipenv creates. Namely, that it stores hashes and does so in one tool. If you wanted to get that from pip, you would need additional tools to get really "frozen" dependencies. Which is possible, but not as easy to use.

Anyway, here is how you can do it: https://notabug.org/ZelphirKaltstahl/python-pipenv-usage/src/master/usage.md

Note, that this script is probably a bit overcomplex, as it takes care of installing pip, virtualenv and pipenv itself with defined hashsums and putting many things into variables. This is done for reproducible results. The only thing still without checksum is setuptools, but you could probably easily copy the approach from the other installed tools.

I recommend trying this script inside a virtual machine first, to understand what it does and only then make use of the parts you need. This is also what I did for testing it. I ran it in a Debian 10 VM.

[kastork]

This isn't intended to be a criticism because I think pipenv is a wonderful tool. But rather to place the framing of this issue back to "feature request" rather than a "how do I...?" string of work-arounds.

I think the point of this issue report is that people have a use-case where pipenv is a fully featured venv management solution (as the name and docs suggest it is); but currently it is a kind of enhanced dependency manager that can work within a venv or will create an opinionated one for its own use as a kind of side-effect. What it does is great, but people (including myself) seem to want a more complete solution that incorporates the really great features of pipenv within a comprehensive venv manager. We're tired of separate env management like conda and workarounds like those suggested by @ZelphirKaltstahl, creating a venv first then managing it with pipenv, and we see the potential that exists in pipenv to fill this role.

A fully realized venv manager would have the ability to specify where the venv files should be located and do what is needed (like maintain the .venv file mentioned in several comments) to keep the project wired to that venv. The --target or .venv directory ideas mentioned by @MegMM and @rr326 seem like intuitive and practical suggestions.

Maybe this notion goes beyond the scope and goals of the project, which is completely fine. If that's the case, then maybe the docs just need to reflect the purpose of the project more clearly so people can easily see that complete management of venvs isn't a use-case pipenv addresses and move on to other solutions.

[ZelphirKaltstahl]

@ZelphirKaltstahl but pipenv should create and activate the virtualenv as necessary, otherwise you way as well just use pip freeze.

That would be nice of course. There is a qualitative difference between pip freeze and a lock file such as Pipenv creates. Namely, that it stores hashes and does so in one tool. If you wanted to get that from pip, you would need additional tools to get really "frozen" dependencies. Which is possible, but not as easy to use.

Anyway, here is how you can do it: https://notabug.org/ZelphirKaltstahl/python-pipenv-usage/src/master/usage.md

Note, that this script is probably a bit overcomplex, as it takes care of installing pip, virtualenv and pipenv itself with defined hashsums and putting many things into variables. This is done for reproducible results. The only thing still without checksum is setuptools, but you could probably easily copy the approach from the other installed tools.

I recommend trying this script inside a virtual machine first, to understand what it does and only then make use of the parts you need. This is also what I did for testing it. I ran it in a Debian 10 VM.

Note, that the script I linked does not seem to work in newer version 2021.11.23 any longer. The newer version has a different question it asks the user:

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Virtualenv already exists!
Remove existing virtualenv? [Y/n]:

If answered with n for "no", Pipenv simply stops doing anything and acts, as if removal of the existing virtualenv was the only option. If you choose to delete the existing venv, then specifying that venv's python3 is no longer existing for use with the --python flag of pipenv install.

It seems, that this workflow is now broken and that truly you cannot use existing virtualenvs any longer for installing your packages in them using Pipenv. If anyone knows another workaround, I'd like to know it.

[matteius]

This is supported now with https://pipenv.pypa.io/en/latest/advanced/#pipenv.environments.Setting.PIPENV_CUSTOM_VENV_NAME

[samweisgamdschie]

This is supported now with https://pipenv.pypa.io/en/latest/advanced/#pipenv.environments.Setting.PIPENV_CUSTOM_VENV_NAME

Works like a charm! Many thanks for that!!

Just a hint, I haven't found this feature in the official docs.

[OrangeDog]

@samweisgamdschie

This document is current in the process of being broken apart into more granular sections so that we may provide better overall documentation.

It's now at https://pipenv.pypa.io/en/latest/configuration/#pipenv.environments.Setting.PIPENV_CUSTOM_VENV_NAME