When using pipenv alongside pyenv, environments created with --python or PIPENV_PYTHON don't use the specified Python version

[stevecshanks]

Issue description

I'm using pipenv alongside pyenv, so followed the instructions at https://pipenv.pypa.io/en/latest/diagnose/#pipenv-does-not-respect-pyenvs-global-and-local-python-versions. But while the specified Python version is used while creating the virtual environment, the Python version inside it is not the same.

I was able to get this working using --python $(pyenv which python) rather than --python $PYENV_ROOT/shims/python - I'm happy to submit a PR updating the docs if that's the right solution.

Expected result

The version of Python inside the virtual env should be the one specified using --python or PIPENV_PYTHON.

Actual result

If my default Python is 3.9, and I specify 3.8, the virtual env created still uses Python 3.9.

The install step below produces the following output:

Using python: /Users/steve/.pyenv/shims/python
Path to python: /Users/steve/.pyenv/shims/python
Creating a virtualenv for this project...
Pipfile: /Users/steve/projects/pipenv-test/Pipfile
Using /Users/steve/.pyenv/shims/python (3.8.0) to create virtualenv...
⠙ Creating virtual environment...created virtual environment CPython3.9.0.final.0-64 in 631ms
  creator CPython3Posix(dest=/Users/steve/.local/share/virtualenvs/pipenv-test-yE2WCZfV, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/steve/Library/Application Support/virtualenv)
    added seed packages: pip==20.2.4, setuptools==50.3.2, wheel==0.35.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment!
Virtualenv location: /Users/steve/.local/share/virtualenvs/pipenv-test-yE2WCZfV
Creating a Pipfile for this project...
Installing requests...
Installing package: requests
Writing supplied requirement line to temporary file: 'requests'
Installing 'requests'
$ ['/Users/steve/.local/share/virtualenvs/pipenv-test-yE2WCZfV/bin/pip', 'install', '--verbose', '--upgrade', '--exists-action=i', '-r', '/var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenv-2m2n8tf6-requirements/pipenv-llj5cqol-requirement.txt', '-i', 'https://pypi.org/simple']
Using source directory: '/Users/steve/.local/share/virtualenvs/pipenv-test-yE2WCZfV/src'
Adding requests to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
⠼ Locking...
                          ROUND 1
⠋ Locking...Current constraints:
  requests (from -r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))

Finding the best candidates:
  found candidate requests==2.25.0 (constraint was <any>)

Finding secondary dependencies:
  requests==2.25.0          requires certifi>=2017.4.17, chardet<4,>=3.0.2, idna<3,>=2.5, urllib3<1.27,>=1.21.1

New dependencies found in this round:
  adding ['certifi', '>=2017.4.17', '[]']
  adding ['chardet', '<4,>=3.0.2', '[]']
  adding ['idna', '<3,>=2.5', '[]']
  adding ['urllib3', '<1.27,>=1.21.1', '[]']
Removed dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  certifi>=2017.4.17 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  chardet<4,>=3.0.2 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  idna<3,>=2.5 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  requests (from -r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  urllib3<1.27,>=1.21.1 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))

Finding the best candidates:
  found candidate certifi==2020.11.8 (constraint was >=2017.4.17)
  found candidate chardet==3.0.4 (constraint was >=3.0.2,<4)
  found candidate idna==2.10 (constraint was >=2.5,<3)
  found candidate requests==2.25.0 (constraint was <any>)
  found candidate urllib3==1.26.2 (constraint was >=1.21.1,<1.27)

Finding secondary dependencies:
  certifi==2020.11.8        requires -
  urllib3==1.26.2           requires -
  idna==2.10                requires -
  chardet==3.0.4            requires -
  requests==2.25.0          requires certifi>=2017.4.17, chardet<4,>=3.0.2, idna<3,>=2.5, urllib3<1.27,>=1.21.1
------------------------------------------------------------
Result of round 2: stable, done

Generating hashes:
  certifi
  urllib3
  idna
  chardet
  requests
ROUND 1
Current constraints:
  requests (from -r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))

Finding the best candidates:
  found candidate requests==2.25.0 (constraint was <any>)

Finding secondary dependencies:
  requests==2.25.0          requires certifi>=2017.4.17, chardet<4,>=3.0.2, idna<3,>=2.5, urllib3<1.27,>=1.21.1

New dependencies found in this round:
  adding ['certifi', '>=2017.4.17', '[]']
  adding ['chardet', '<4,>=3.0.2', '[]']
  adding ['idna', '<3,>=2.5', '[]']
  adding ['urllib3', '<1.27,>=1.21.1', '[]']
Removed dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  certifi>=2017.4.17 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  chardet<4,>=3.0.2 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  idna<3,>=2.5 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  requests (from -r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))
  urllib3<1.27,>=1.21.1 (from requests==2.25.0->-r /var/folders/4s/_vl6qk7x14lb28jn6yh9z76r0000gn/T/pipenvzf06t399requirements/pipenv-57vc1sxd-constraints.txt (line 2))

Finding the best candidates:
  found candidate certifi==2020.11.8 (constraint was >=2017.4.17)
  found candidate chardet==3.0.4 (constraint was >=3.0.2,<4)
  found candidate idna==2.10 (constraint was >=2.5,<3)
  found candidate requests==2.25.0 (constraint was <any>)
  found candidate urllib3==1.26.2 (constraint was >=1.21.1,<1.27)

Finding secondary dependencies:
  certifi==2020.11.8        requires -
  urllib3==1.26.2           requires -
  idna==2.10                requires -
  chardet==3.0.4            requires -
  requests==2.25.0          requires certifi>=2017.4.17, chardet<4,>=3.0.2, idna<3,>=2.5, urllib3<1.27,>=1.21.1
------------------------------------------------------------
Result of round 2: stable, done

Generating hashes:
  certifi
  urllib3
  idna
  chardet
  requests
✔ Success!
Updated Pipfile.lock (fe5a22)!
Installing dependencies from Pipfile.lock (fe5a22)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Steps to replicate

pyenv local 3.8.0
pipenv install requests --verbose --python $PYENV_ROOT/shims/python
pipenv run python --version

---

$ pipenv --support

Pipenv version: '2020.11.4'

Pipenv location: '/usr/local/Cellar/pipenv/2020.11.4/libexec/lib/python3.9/site-packages/pipenv'

Python location: '/usr/local/Cellar/pipenv/2020.11.4/libexec/bin/python'

Python installations found:

• 3.9.0: /Users/steve/.pyenv/versions/3.9.0/bin/python3
• 3.8.0: /Users/steve/.pyenv/versions/3.8.0/bin/python3
• 3.6.1: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
• 3.6.1: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6
• 3.6.1: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6m
• 3.6.1: /usr/local/bin/python3
• 3.6.1: /usr/local/bin/python3.6
• 3.6.1: /usr/local/bin/python3.6m
• 2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.9.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.7.0: Mon Aug 31 22:11:23 PDT '
                     '2020; root:xnu-4570.71.82.6~1/RELEASE_X86_64',
 'python_full_version': '3.9.0',
 'python_version': '3.9',
 'sys_platform': 'darwin'}

System environment variables:

• PATH
• TERM_PROGRAM
• PYENV_ROOT
• TERM
• SHELL
• TMPDIR
• Apple_PubSub_Socket_Render
• TERM_PROGRAM_VERSION
• TERM_SESSION_ID
• ZSH
• USER
• SSH_AUTH_SOCK
• __CF_USER_TEXT_ENCODING
• PAGER
• LSCOLORS
• _
• PWD
• LANG
• ITERM_PROFILE
• XPC_FLAGS
• XPC_SERVICE_NAME
• PYENV_SHELL
• SHLVL
• HOME
• COLORFGBG
• LC_TERMINAL_VERSION
• ITERM_SESSION_ID
• LESS
• LOGNAME
• LC_CTYPE
• GOPATH
• LC_TERMINAL
• COLORTERM
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/Cellar/pipenv/2020.11.4/libexec/tools:/Users/steve/.pyenv/shims:/Users/steve/projects/go/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin
• SHELL: /bin/zsh
• LANG: en_GB.UTF-8
• PWD: /Users/steve/projects/pipenv-test

---

Contents of Pipfile ('/Users/steve/projects/pipenv-test/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
requests = "*"

[dev-packages]

[requires]
python_version = "3.9"

Contents of Pipfile.lock ('/Users/steve/projects/pipenv-test/Pipfile.lock'):

{
  "_meta": {
    "hash": {
      "sha256": "b8c2e1580c53e383cfe4254c1f16560b855d984fde8b2beb3bf6ee8fc2fe5a22"
    },
    "pipfile-spec": 6,
    "requires": {
      "python_version": "3.9"
    },
    "sources": [
      {
        "name": "pypi",
        "url": "https://pypi.org/simple",
        "verify_ssl": true
      }
    ]
  },
  "default": {
    "certifi": {
      "hashes": [
        "sha256:1f422849db327d534e3d0c5f02a263458c3955ec0aae4ff09b95f195c59f4edd",
        "sha256:f05def092c44fbf25834a51509ef6e631dc19765ab8a57b4e7ab85531f0a9cf4"
      ],
      "version": "==2020.11.8"
    },
    "chardet": {
      "hashes": [
        "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
        "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
      ],
      "version": "==3.0.4"
    },
    "idna": {
      "hashes": [
        "sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
        "sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
      ],
      "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
      "version": "==2.10"
    },
    "requests": {
      "hashes": [
        "sha256:7f1a0b932f4a60a1a65caa4263921bb7d9ee911957e0ae4a23a6dd08185ad5f8",
        "sha256:e786fa28d8c9154e6a4de5d46a1d921b8749f8b74e28bde23768e5e16eece998"
      ],
      "index": "pypi",
      "version": "==2.25.0"
    },
    "urllib3": {
      "hashes": [
        "sha256:19188f96923873c92ccb987120ec4acaa12f0461fa9ce5d3d0772bc965a39e08",
        "sha256:d8ff90d979214d7b4f8ce956e80f4028fc6860e4431f731ea4a8c08f23f99473"
      ],
      "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
      "version": "==1.26.2"
    }
  },
  "develop": {}
}

[frostming]

Can't reproduce with exactly the same steps in a linuxbrew image:

linuxbrew@037a2aea3da9:~/app$ pyenv global 3.9.0
linuxbrew@037a2aea3da9:~/app$ python -V
Python 3.9.0
linuxbrew@037a2aea3da9:~/app$ pyenv local 3.8.6
linuxbrew@037a2aea3da9:~/app$ pipenv --python ~/.pyenv/shims/python
Virtualenv already exists!
Removing existing virtualenv...
Warning: the environment variable LANG is not set!
We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.
Creating a virtualenv for this project...
Pipfile: /home/linuxbrew/app/Pipfile
Using /home/linuxbrew/.pyenv/shims/python (3.8.6) to create virtualenv...
⠴ Creating virtual environment...created virtual environment CPython3.8.6.final.0-64 in 287ms
  creator CPython3Posix(dest=/home/linuxbrew/.local/share/virtualenvs/app-F-L2ItSf, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/linuxbrew/.local/share/virtualenv)
    added seed packages: pip==20.2.4, setuptools==50.3.2, wheel==0.35.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/linuxbrew/.local/share/virtualenvs/app-F-L2ItSf

[frostming]

But basically, avoid using shimmed Python interpreter to create virtualenv

[stevecshanks]

I've been able to replicate on another Mac, but was struggling to get Python to build correctly on linuxbrew, so wasn't able to try your steps above.

I agree that avoiding using the shim should fix the issue though, so I've pushed up a documentation PR: #4535

Happy to make any changes to the above, just let me know!