Configuration option to force use of system site-packages

[jonapich]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

We need a way to tell poetry to ignore the currently active virtual environment. I am running into a roadblock trying to automate some of the poetry install operations from python.

As a comparison, pipenv supports it through the PIPENV_IGNORE_VIRTUALENVS environment variable.

The best approach, I think, is a switch like --breakout-of-venv (can't find a good name) to tell poetry to ignore the currently activated environment for any command that needs to deal with the virtual environment (install, add, remove...).

Other ideas:

• It feels like poetry env is the right candidate for this. Something like poetry env use --default maybe. But then you always need 2 commands per poetry command if you're using e.g. subprocess.call() to make it work; only useful in a real CLI context.
• A poetry config could work, but it sounds weird to modify this in order to run a script.
• An env variable -only solution like pipenv does is cumbersome and annoying to use, but having the option to do it is useful for CI servers.

[znd4]

Are there any workarounds for this atm? What kind of work would be required to implement this?

For some input on the proposed interface for this, I really like the environment variable approach because it can be very temporary (it can be used for the duration of a single subprocess call

[finswimmer]

Hello @jonapich and @zdog234,

could you please describe your use-cases? For now it's hard for me to imagine why this feature would be needed.

fin swimmer

[jonapich]

CI servers may want to launch poetry for various reasons. The "thing" that launches poetry, may be a Python script launched from another virtual environment.

Another use case is to be able to script poetry actions in a batch. I have several small python projects in 1 tree and I like to automate common operations. Because of poetry's limitation, I was forced to do it in bash instead of python.

[znd4]

Sure thing! I'm not actually sure if this particular issue would solve the problems I've had, or if I could write an issue that's more specific to windows.

I've got a library called scan (fake name, this is an internal tool) that could be running in either python 3.6, 3.7, or 3.8.

scan needs to be able to subprocess.run("poetry install") on any pyproject.toml w/ python 3.6, 3.7 or 3.8. This is pretty easy to get working in linux with pyenv, but in windows, even if I've got the right python versions installed, and python3.6, python3.7, and python3.8 shortcuts created and in my path, poetry is hanging up on only finding the currently activated virtual environment (the one that scan is installed in).

I'd have to go verify this behavior, but I'm pretty sure that poetry env use wasn't fixing the problem either -- even if I called subprocess.run("poetry env use C:\path\to\python3.6") in the pyproject.toml's parent directory, poetry only tried to use the activated virtual environment when I later invoked poetry install, which would throw an error because scan was installed into a python3.8 virtual environment and the pyproject.toml required 3.6

We tried using pyenv-win as a solution, but after a few hours of trying that, we decided it was easier for all of our developers to install WSL (I wish we just all had linux laptops anyways, but old companies love windows)

[znd4]

Ah, sorry for that long wall of text. Just realized that my answer is covered by @jonapich 's

[jonapich]

I was able to work around it by removing that env variable before launching poetry through a subprocess.call:

            environment_variables.pop('VIRTUAL_ENV', None)

[roxchkplusony]

Same use case here as @jonapich. We run our tests via scripts in jenkins and they start with some default virtual environment. That's fine and dandy for anything not poetry, but for poetry I'd like it to be using separate venvs. I'm working around it now by creating and activating venvs manually for each project involved in the one script.

[sinoroc]

If I may...

In particular for the CI workflows, are you sure you need poetry's CLI? Which kinds of poetry commands are you running in those 3rd party environments?

[roxchkplusony]

poetry install before poetry run pytest. We need the pythonpath and packages to be correct in order to run the tests.

[sinoroc]

@roxchkplusony Thanks.

I would guess, that the blocking bit, is that you rely on the dev-dependencies defined in pyproject.toml. Otherwise you could do all this without any poetry command. Does that seem accurate?
Strike that, I think I got the problem the other way around in my head.

[znd4]

@sinoroc, for what it's worth, I need to:

1. Create a virtual environment with the right python version pulled from the pyproject.toml
2. Pip install all dependencies into that virtual environment (--dev included)

Everything else could probably be easily automated.

I guess we could parse out the python version specification to meet 1, but it doesn't seem fun to parse the version

[sinoroc]

@zdog234

scan needs to be able to subprocess.run("poetry install") on any pyproject.toml w/ python 3.6, 3.7 or 3.8. This is pretty easy to get working in linux with pyenv, but in windows, even if I've got the right python versions installed, and python3.6, python3.7, and python3.8 shortcuts created and in my path, poetry is hanging up on only finding the currently activated virtual environment (the one that scan is installed in).

I think I would use tox for a use case like that.

[sinoroc]

Also I do not understand why the following does not execute in the environment created by poetry (i.e. not the one currently active)?

VIRTUAL_ENV= poetry run python -m site

I had understood that poetry relies on the VIRTUAL_ENV environment variable to detect if it is running in a virtual environment, but it might be a bit more complicated than that.

[znd4]

@zdog234

scan needs to be able to subprocess.run("poetry install") on any pyproject.toml w/ python 3.6, 3.7 or 3.8. This is pretty easy to get working in linux with pyenv, but in windows, even if I've got the right python versions installed, and python3.6, python3.7, and python3.8 shortcuts created and in my path, poetry is hanging up on only finding the currently activated virtual environment (the one that scan is installed in).

I think I would use tox for a use case like that.

I think you're misunderstanding my (poorly written) comment. One of the assumptions of my script is that there are some (user-provided) pyproject.toml s, each of which could require python3.6, 3.7, or 3.8

[znd4]

I guess I'd throw 3.9 in now as well

[sinoroc]

I think you're misunderstanding my (poorly written) comment. One of the assumptions of my script is that there are some (user-provided) pyproject.toml s, each of which could require python3.6, 3.7, or 3.8

I see. My bad :/

[znd4]

Np :)

[venaturum]

I have this issue too.

We use a CLI package at work to automate project setup.

It downloads a repo (which contains a pyproject.toml), copies githooks to the appropriate folder, uses pyenv to download the python version pinned in pyproject.toml, sets nbdime to be the default diff viewer for notebooks, and finally uses poetry to set virtualenvs.create true and install the dependencies.

If the CLI package is installed in base python then everything works. If the CLI package is installed and ran in it's own virtual environment (as it should be), then Poetry installs the dependencies into the CLI package's virtual environment, which is unwanted.

[principalideal0]

I was able to work around it by removing that env variable before launching poetry through a subprocess.call:

            environment_variables.pop('VIRTUAL_ENV', None)

Regrettably this does not work for me at all. I'm really pulling my hair out here - I don't know why Poetry defaults to this behavior?