Pipenv 2020.6.2 Falsly flags Homebrew's Python as a virtual environment

[nstapelbroek]

Hey Hey 👋,

First off. Thank you for all the work that has gone into the recent releases! I'm happy to use pipenv almost on a daily basis.

Issue description

It seems that the latest pipenv falsely flags my macOS + Python3 via Homebrew as a virtual environment.

The subsequent installation of dependencies does noting on existing projects or fails weirdly with python2 errors on new projects.

Expected result

$ pipenv install

Warning: the environment variable LANG is not set!
We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.
Creating a virtualenv for this project…
Pipfile: /private/tmp/cool-project/Pipfile
Using /usr/local/bin/python3.7m (3.7.7) to create virtualenv…
⠴ Creating virtual environment...Using base prefix '/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python3.7
Also creating executable in /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/local/bin/python3.7m

✔ Successfully created virtual environment!
Virtualenv location: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz
Installing dependencies from Pipfile.lock (de0b8f)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 21/21 — 00:00:02
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Actual result on new projects

$ pipenv install

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (de0b8f)…
Failed to load paths: /bin/sh: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python: No such file or directory

Output:
Failed to load paths: /bin/sh: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python: No such file or directory

Output:
Failed to load paths: /bin/sh: /Users/nico/.local/share/virtualenvs/cool-project-gb-P45lz/bin/python: No such file or directory

Output:
An error occurred while installing cachetools==4.1.0 --hash=sha256:1d057645db16ca7fe1f3bd953558897603d6f0b9c51ed9d11eb4d071ec4e2aab --hash=sha256:de5d88f87781602201cde465d3afe837546663b168e8b39df67411b0bf10cefc! Will try again.
An error occurred while installing marshmallow==3.6.0 --hash=sha256:f88fe96434b1f0f476d54224d59333eba8ca1a203a2695683c1855675c4049a7 --hash=sha256:c2673233aa21dde264b84349dc2fd1dce5f30ed724a0a00e75426734de5b84ab! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 21/21 — 00:00:02
Installing initially failed dependencies…
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 252, in install
[InstallError]:       site_packages=state.site_packages
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2062, in do_install
[InstallError]:       keep_outdated=keep_outdated
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1314, in do_init
[InstallError]:       pypi_mirror=pypi_mirror,
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 901, in do_install_dependencies
[InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement cachetools==4.1.0 (from -r /var/folders/bm/nrklr36j2d92m7wz22_1nkym0000gn/T/pipenv-lc3jfmuu-requirements/pipenv-nnfotjey-requirement.txt (line 1)) (from versions: 0.0.0, 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 0.5.0, 0.5.1, 0.6.0, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.8.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 2.0.0, 2.0.1, 2.1.0, 3.0.0, 3.1.0, 3.1.1)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for cachetools==4.1.0 (from -r /var/folders/bm/nrklr36j2d92m7wz22_1nkym0000gn/T/pipenv-lc3jfmuu-requirements/pipenv-nnfotjey-requirement.txt (line 1))
ERROR: Couldn't install package: cachetools
 Package installation failed...
  ☤  ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/2 —

Actual result on existing projects

$ pipenv install

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (de0b8f)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 —

Steps to replicate

1. Use macOS (I'm on Mojave 10.14.6)
2. Install python using homebrew: brew install python
   a. Probably unrelated: I've also installed Python 3.8 using homebrew brew install python@3.8
3. Also make sure the install paths of homebrew are in your PATH.
   a. I think my setup is based upon this guide
   b. Make sure to reload/reboot any open sessions so the changes in PATH are reflected
4. Run pip3 install -U pipenv to install the latest pipenv
5. Run pipenv install in a python project
6. Acknowledge incorrect virtual environment detection because it spits out the "Pipenv found itself running within a virtual environment" message.

System info

$ pipenv --support

Pipenv version: '2020.6.2'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

• 3.7.7: /usr/local/bin/python3
• 3.7.7: /usr/local/bin/python3.7m
• 3.7.7: /usr/local/bin/python3.7
• 3.6.7: /usr/local/bin/python3.6
• 3.6.7: /usr/local/bin/python3.6m
• 2.7.17: /usr/local/bin/python2
• 2.7.17: /usr/local/bin/python2.7
• 2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.7',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '18.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT '
                     '2020; root:xnu-4903.278.35~1/RELEASE_X86_64',
 'python_full_version': '3.7.7',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

• TERM_SESSION_ID
• SSH_AUTH_SOCK
• LC_TERMINAL_VERSION
• Apple_PubSub_Socket_Render
• COLORFGBG
• ITERM_PROFILE
• XPC_FLAGS
• PWD
• SHELL
• LC_CTYPE
• TERM_PROGRAM_VERSION
• TERM_PROGRAM
• PATH
• LC_TERMINAL
• COLORTERM
• TERM
• HOME
• TMPDIR
• USER
• XPC_SERVICE_NAME
• LOGNAME
• ITERM_SESSION_ID
• __CF_USER_TEXT_ENCODING
• SHLVL
• OLDPWD
• ZSH
• PAGER
• LESS
• LSCOLORS
• ZNT_REPO_DIR
• ZNT_CONFIG_DIR
• NVM_DIR
• GOPATH
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/opt/openjdk@11/bin:/Users/nico/Library/Android/sdk/platform-tools:/Users/nico/Library/Python/3.7/bin:/Users/nico/Projects/go/bin:/usr/local/bin:/usr/local/sbin:/Users/nico/.poetry/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/local/CrossPack-AVR/bin:/Users/nico/.composer/vendor/bin
• SHELL: /bin/zsh
• PWD: /tmp/cool-project

---

Contents of Pipfile ('/private/tmp/cool-project/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
flask = "*"
docker = "*"
waitress = "*"
flask-basicauth = "*"
sentry-sdk = {extras = ["flask"],version = "*"}
flask-marshmallow = "*"
cachetools = "*"

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/private/tmp/cool-project/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "09c8342a4c3edee1ba3cffcf27e6d97613ed3c020e1c1c77b62a9fd7ffde0b8f"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "blinker": {
            "hashes": [
                "sha256:471aee25f3992bd325afa3772f1063dbdbbca947a041b8b89466dc00d606f8b6"
            ],
            "version": "==1.4"
        },
        "cachetools": {
            "hashes": [
                "sha256:1d057645db16ca7fe1f3bd953558897603d6f0b9c51ed9d11eb4d071ec4e2aab",
                "sha256:de5d88f87781602201cde465d3afe837546663b168e8b39df67411b0bf10cefc"
            ],
            "index": "pypi",
            "version": "==4.1.0"
        },
        "certifi": {
            "hashes": [
                "sha256:1d987a998c75633c40847cc966fcf5904906c920a7f17ef374f5aa4282abd304",
                "sha256:51fcb31174be6e6664c5f69e3e1691a2d72a1a12e90f872cbdb1567eb47b6519"
            ],
            "version": "==2020.4.5.1"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "click": {
            "hashes": [
                "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
                "sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
            ],
            "version": "==7.1.2"
        },
        "docker": {
            "hashes": [
                "sha256:1c2ddb7a047b2599d1faec00889561316c674f7099427b9c51e8cb804114b553",
                "sha256:ddae66620ab5f4bce769f64bcd7934f880c8abe6aa50986298db56735d0f722e"
            ],
            "index": "pypi",
            "version": "==4.2.0"
        },
        "flask": {
            "hashes": [
                "sha256:4efa1ae2d7c9865af48986de8aeb8504bf32c7f3d6fdc9353d34b21f4b127060",
                "sha256:8a4fdd8936eba2512e9c85df320a37e694c93945b33ef33c89946a340a238557"
            ],
            "index": "pypi",
            "version": "==1.1.2"
        },
        "flask-basicauth": {
            "hashes": [
                "sha256:df5ebd489dc0914c224419da059d991eb72988a01cdd4b956d52932ce7d501ff"
            ],
            "index": "pypi",
            "version": "==0.2.0"
        },
        "flask-marshmallow": {
            "hashes": [
                "sha256:6e6aec171b8e092e0eafaf035ff5b8637bf3a58ab46f568c4c1bab02f2a3c196",
                "sha256:a1685536e7ab5abdc712bbc1ac1a6b0b50951a368502f7985e7d1c27b3c21e59"
            ],
            "index": "pypi",
            "version": "==0.12.0"
        },
        "idna": {
            "hashes": [
                "sha256:7588d1c14ae4c77d74036e8c22ff447b26d0fde8f007354fd48a7814db15b7cb",
                "sha256:a068a21ceac8a4d63dbfd964670474107f541babbd2250d61922f029858365fa"
            ],
            "version": "==2.9"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
                "sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
            ],
            "version": "==2.11.2"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
                "sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
            ],
            "version": "==1.1.1"
        },
        "marshmallow": {
            "hashes": [
                "sha256:c2673233aa21dde264b84349dc2fd1dce5f30ed724a0a00e75426734de5b84ab",
                "sha256:f88fe96434b1f0f476d54224d59333eba8ca1a203a2695683c1855675c4049a7"
            ],
            "version": "==3.6.0"
        },
        "requests": {
            "hashes": [
                "sha256:43999036bfa82904b6af1d99e4882b560e5e2c68e5c4b0aa03b655f3d7d73fee",
                "sha256:b3f43d496c6daba4493e7c431722aeb7dbc6288f52a6e04e7b6023b0247817e6"
            ],
            "version": "==2.23.0"
        },
        "sentry-sdk": {
            "extras": [
                "flask"
            ],
            "hashes": [
                "sha256:0e5e947d0f7a969314aa23669a94a9712be5a688ff069ff7b9fc36c66adc160c",
                "sha256:799a8bf76b012e3030a881be00e97bc0b922ce35dde699c6537122b751d80e2c"
            ],
            "index": "pypi",
            "version": "==0.14.4"
        },
        "six": {
            "hashes": [
                "sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
                "sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
            ],
            "version": "==1.14.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527",
                "sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115"
            ],
            "version": "==1.25.9"
        },
        "waitress": {
            "hashes": [
                "sha256:045b3efc3d97c93362173ab1dfc159b52cfa22b46c3334ffc805dbdbf0e4309e",
                "sha256:77ff3f3226931a1d7d8624c5371de07c8e90c7e5d80c5cc660d72659aaf23f38"
            ],
            "index": "pypi",
            "version": "==1.4.3"
        },
        "websocket-client": {
            "hashes": [
                "sha256:0fc45c961324d79c781bab301359d5a1b00b13ad1b10415a4780229ef71a5549",
                "sha256:d735b91d6d1692a6a181f2a8c9e0238e5f6373356f561bb9dc4c7af36f452010"
            ],
            "version": "==0.57.0"
        },
        "werkzeug": {
            "hashes": [
                "sha256:2de2a5db0baeae7b2d2664949077c2ac63fbd16d98da0ff71837f7d1dea3fd43",
                "sha256:6c80b1e5ad3665290ea39320b91e1be1e0d5f60652b964a3070216de83d2e47c"
            ],
            "version": "==1.0.1"
        }
    },
    "develop": {}
}

Notes

Assuming that a Homebrew installed python is not supposed to be a virtual environment, I tried digging into this myself and found a possible cause of the bug. Please note that i'm making some assumptions so I'd love to hear your input on this :)

The last release introduced an _OSX_VENV variable that is later-on used in the method is_in_virtualenv() to determine if the command is running inside
a virtual environment. This variable stores the removed value of an __PYVENV_LAUNCHER__ environment variable, which is originally implemented to prevent faulty shebangs.

I suspect that using the OSX_VENV variable is too aggressive in determining if we are running inside a virtual environment. I've checked on a different mac setup with a Python 3.7.2 and it seems that the __PYVENV_LAUNCHER__ is also set there.

My suggested fix would be to remove the _OSX_VENV variable, this works on my machine. Although I'm not sure if this reintroduces #4284

Edit: Formatting and typo's

[ewehner]

Just a note: this was closed as a duplicate (#4341) but I think it adds something to this issue... I'm also experiencing this issue, and to me it seems 2-fold. First, it throws the warning about the existing environment when there is one, but it still will proceed to successfully create a pipenv. However, pipenv --rm will throw the same error and then abort the removal. So pipenv --rm is broken in this case, and the virtualenv must be removed manually

[goude]

Seeing this too. Looks like #4341 was closed as a duplicate of this issue?

[TimidRobot]

To manually remove a pipenv virtualenv:

pipenv_venv="$(PIPENV_IGNORE_VIRTUALENVS=1 pipenv --venv)" && rm -rf "${pipenv_venv}"

[nstapelbroek]

Hey, I noticed that this is reopened. Did the applied fix in #4317 not work well enough?

I read #4359 but can only assume why this reopened 😄

[frostming]

@nstapelbroek Just to leave at least one issue open so that newcomers won't file duplicate issues.

I didn't notice #4359 , I have closed that one.

[J-M0]

I'm also experiencing this on Red Hat 7 when I install pipenv with pipx and on macOS via brew . From what I can tell though, pipenv install and pipenv --rm work as expected if pipenv was not installed in a virtualenv and the message is incorrect about using the virtualenv pipenv is installed in.

This also still happens when I install pipenv from the master branch.