Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache invalidation broken with pyenv? #2130

Closed
cjolowicz opened this issue Jun 9, 2021 · 3 comments
Closed

Cache invalidation broken with pyenv? #2130

cjolowicz opened this issue Jun 9, 2021 · 3 comments
Labels
bug help-wanted

Comments

@cjolowicz
Copy link
Contributor

cjolowicz commented Jun 9, 2021 •
edited
Loading

Issue

After installing a new Python point release X.Y.Z+1 using pyenv on Mac, virtualenv -p X.Y still uses the previous release from its cache X.Y.Z.

This issue seems specific to Mac, I could not reproduce this on Linux.

Update: I have just reproduced this on Linux, as well. See repro below.

Looking at the cache invalidation logic in virtualenv.discovery.cached_py_info._get_via_file_cache, the cause seems to be that the mtime of the pyenv shim for pythonX.Y does not change when a newer version is installed.

To reproduce, assuming a Mac with pyenv and virtualenv installed:

pyenv install 3.7.9
pyenv global 3.7.9
virtualenv -p 3.7 /tmp/venv-old # uses 3.7.9

pyenv install 3.7.10
pyenv global 3.7.10
virtualenv -p 3.7 /tmp/venv-new  # uses 3.7.9

touch ~/.pyenv/shims/python3.7
virtualenv -p 3.7 /tmp/venv  # uses 3.7.10

The first virtualenv invocation adds 3.7.9 to the cache. The second invocation still uses 3.7.9 for the environment, even though 3.7.10 is activated. The mtime of the shim for python3.7 never changes (even when doing a pyenv rehash).

Expand for the full session with some debug statements inserted in _get_via_file_cache 
$ virtualenv --version
virtualenv 20.4.7 from /private/tmp/v/lib/python3.8/site-packages/virtualenv/__init__.py

$ pyenv global 3.7.9

$ python -c "import pathlib; print(pathlib.Path('/Users/cjolowicz/.pyenv/shims/python3.7').stat().st_mtime)"
1623268689.907189

$ virtualenv -p 3.7 /tmp/v-3.7-1
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/private/tmp/v/bin/python3')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/private/tmp/v/bin/python3'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/shims/python3.7')
_get_via_file_cache: path_modified = 1623268689.907189
_get_via_file_cache: store: path_modified = 1623268689.907189
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7')
_get_via_file_cache: path_modified = 1599937426.8033798
_get_via_file_cache: store: path_modified = 1599937426.8033798
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
created virtual environment CPython3.7.9.final.0-64 in 2474ms
  creator CPython3Posix(dest=/private/tmp/v-3.7-1, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/cjolowicz/Library/Application Support/virtualenv)
    added seed packages: pip==21.1.1, setuptools==56.2.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

$ pyenv install 3.7.10
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.10.tar.xz...
-> https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tar.xz
Installing Python-3.7.10...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.7.10 to /Users/cjolowicz/.pyenv/versions/3.7.10

$ python -c "import pathlib; print(pathlib.Path('/Users/cjolowicz/.pyenv/shims/python3.7').stat().st_mtime)"
1623268689.907189

$ virtualenv -p 3.7 /tmp/v-3.7-2
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/private/tmp/v/bin/python3')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/private/tmp/v/bin/python3'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/shims/python3.7')
_get_via_file_cache: path_modified = 1623268689.907189
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/shims/python3.7'
_get_via_file_cache: of_st_mtime = 1623268689.907189
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7')
_get_via_file_cache: path_modified = 1599937426.8033798
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7'
_get_via_file_cache: of_st_mtime = 1599937426.8033798
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
created virtual environment CPython3.7.9.final.0-64 in 1102ms
  creator CPython3Posix(dest=/private/tmp/v-3.7-2, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/cjolowicz/Library/Application Support/virtualenv)
    added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

$ pyenv rehash

$ python -c "import pathlib; print(pathlib.Path('/Users/cjolowicz/.pyenv/shims/python3.7').stat().st_mtime)"
1623268689.907189

$ pyenv global
3.7.9

$ pyenv global 3.7.10

$ python -c "import pathlib; print(pathlib.Path('/Users/cjolowicz/.pyenv/shims/python3.7').stat().st_mtime)"
1623268689.907189

$ virtualenv -p 3.7 /tmp/v-3.7-3
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/private/tmp/v/bin/python3')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/private/tmp/v/bin/python3'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/shims/python3.7')
_get_via_file_cache: path_modified = 1623268689.907189
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/shims/python3.7'
_get_via_file_cache: of_st_mtime = 1623268689.907189
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7')
_get_via_file_cache: path_modified = 1599937426.8033798
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7'
_get_via_file_cache: of_st_mtime = 1599937426.8033798
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
created virtual environment CPython3.7.9.final.0-64 in 1167ms
  creator CPython3Posix(dest=/private/tmp/v-3.7-3, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/cjolowicz/Library/Application Support/virtualenv)
    added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

$ touch ~/.pyenv/shims/python3.7

$ virtualenv -p 3.7 /tmp/v-3.7-4
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/private/tmp/v/bin/python3')
_get_via_file_cache: path_modified = 1601133440.6395226
_get_via_file_cache: of_path = '/private/tmp/v/bin/python3'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/shims/python3.7')
_get_via_file_cache: path_modified = 1623274382.798455
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/shims/python3.7'
_get_via_file_cache: of_st_mtime = 1623268689.907189
_get_via_file_cache: store: path_modified = 1623274382.798455
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=10, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.7.10/bin/python3.7')
_get_via_file_cache: path_modified = 1623271004.5423782
_get_via_file_cache: store: path_modified = 1623271004.5423782
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=10, releaselevel='final', serial=0)
created virtual environment CPython3.7.10.final.0-64 in 1129ms
  creator CPython3Posix(dest=/private/tmp/v-3.7-4, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/cjolowicz/Library/Application Support/virtualenv)
    added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

Here is a reproduction for Linux using Docker:

FROM debian:stable-slim
ARG DEBIAN_FRONTEND=noninteractive

# Install system Python (3.7.3) and latest virtualenv.
RUN set -ex; apt-get update; apt-get install -y python3 python3-pip
RUN set -ex; pip3 install --user virtualenv
ENV PATH /root/.local/bin:$PATH

# Install Python build requirements for pyenv.
RUN set -ex; apt-get update; apt-get install -y \
    build-essential \
    ca-certificates \
    curl \
    git \
    libbz2-dev \
    libffi-dev \
    liblzma-dev \
    libncursesw5-dev \
    libreadline-dev \
    libsqlite3-dev \
    libssl-dev \
    libxml2-dev \
    libxmlsec1-dev \
    llvm \
    tk-dev \
    wget \
    xz-utils \
    zlib1g-dev

# Install pyenv.
RUN curl https://pyenv.run | bash
ENV PYENV_ROOT /root/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH

# Ensure pyenv is loaded on each RUN.
RUN >> /root/pyenv.sh echo '#!/bin/bash'
RUN >> /root/pyenv.sh echo 'eval "$(pyenv init -)"'
RUN >> /root/pyenv.sh echo 'eval "$(pyenv virtualenv-init -)"'
RUN >> /root/pyenv.sh echo 'exec /bin/bash -c "$*"'
RUN chmod a+x /root/pyenv.sh
SHELL ["/root/pyenv.sh"]

# Install Python 3.9.4 and create a virtualenv.
RUN pyenv install 3.9.4 && pyenv global 3.9.4
RUN date --reference=/root/.pyenv/shims/python3.9 --utc --iso-8601=ns
RUN virtualenv --python=3.9 /tmp/venv-old
RUN /tmp/venv-old/bin/python -VV

# Install Python 3.9.5 and create a virtualenv.
RUN pyenv install 3.9.5 && pyenv global 3.9.5
RUN date --reference=/root/.pyenv/shims/python3.9 --utc --iso-8601=ns
RUN virtualenv --python=3.9 /tmp/venv-new
RUN /tmp/venv-new/bin/python -VV

Environment

Provide at least:

  • OS: Mac (Darwin XYZ-MBP 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64 i386 MacBookPro14,1 Darwin)

  • pip list of the host python where virtualenv is installed:

    Package    Version
---------- -------
appdirs    1.4.4
distlib    0.3.2
filelock   3.0.12
pip        21.1.2
setuptools 56.2.0
six        1.16.0
virtualenv 20.4.7
wheel      0.36.2

Output of the virtual environment creation

Make sure to run the creation with `-vvv --with-traceback`: 
109 setup logging to NOTSET [DEBUG report:43]
119 find interpreter for spec PythonSpec(major=3, minor=7) [INFO builtin:62]
119 discover exe for PythonInfo(spec=CPython3.8.6.final.0-64, exe=/private/tmp/v/bin/python, platform=darwin, version='3.8.6 (default, Sep 26 2020, 17:16:54) \n[Clang 11.0.0 (clang-1100.0.20.17)]', encoding_fs_io=utf-8-utf-8) in /Users/cjolowicz/.pyenv/versions/3.8.6 [DEBUG py_info:372]
122 filesystem is not case-sensitive [DEBUG info:28]
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8')
_get_via_file_cache: path_modified = 1601133440.6395226
125 Attempting to acquire lock 4348807968 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/4351eb5e2327093fb909053c72b2c4b3a73ed4c74a9c9913244f83bb0e99c322.lock [DEBUG filelock:270]
126 Lock 4348807968 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/4351eb5e2327093fb909053c72b2c4b3a73ed4c74a9c9913244f83bb0e99c322.lock [INFO filelock:274]
126 got python info of /Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8 from /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/4351eb5e2327093fb909053c72b2c4b3a73ed4c74a9c9913244f83bb0e99c322.json [DEBUG via_disk_folder:135]
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
127 Attempting to release lock 4348807968 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/4351eb5e2327093fb909053c72b2c4b3a73ed4c74a9c9913244f83bb0e99c322.lock [DEBUG filelock:315]
135 Lock 4348807968 released on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/4351eb5e2327093fb909053c72b2c4b3a73ed4c74a9c9913244f83bb0e99c322.lock [INFO filelock:318]
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
136 proposed PythonInfo(spec=CPython3.8.6.final.0-64, system=/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8, exe=/private/tmp/v/bin/python, platform=darwin, version='3.8.6 (default, Sep 26 2020, 17:16:54) \n[Clang 11.0.0 (clang-1100.0.20.17)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:69]
136 discover PATH[0]=/private/tmp/v/bin [DEBUG builtin:114]
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/private/tmp/v/bin/python3')
_get_via_file_cache: path_modified = 1601133440.6395226
137 Attempting to acquire lock 4348808688 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/c746f01974fd30bc97702678e530f7dac533b1246225a9dce3a7cecb9950f535.lock [DEBUG filelock:270]
137 Lock 4348808688 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/c746f01974fd30bc97702678e530f7dac533b1246225a9dce3a7cecb9950f535.lock [INFO filelock:274]
138 got python info of /private/tmp/v/bin/python3 from /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/c746f01974fd30bc97702678e530f7dac533b1246225a9dce3a7cecb9950f535.json [DEBUG via_disk_folder:135]
_get_via_file_cache: of_path = '/private/tmp/v/bin/python3'
_get_via_file_cache: of_st_mtime = 1601133440.6395226
138 Attempting to release lock 4348808688 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/c746f01974fd30bc97702678e530f7dac533b1246225a9dce3a7cecb9950f535.lock [DEBUG filelock:315]
138 Lock 4348808688 released on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/c746f01974fd30bc97702678e530f7dac533b1246225a9dce3a7cecb9950f535.lock [INFO filelock:318]
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)
138 discover exe from cache /Users/cjolowicz/.pyenv/versions/3.8.6 - exact False: PythonInfo({'architecture': 64, 'base_exec_prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'base_prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'distutils_install': {'data': '', 'headers': 'include/python3.8/UNKNOWN', 'platlib': 'lib/python3.8/site-packages', 'purelib': 'lib/python3.8/site-packages', 'scripts': 'bin'}, 'exec_prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'executable': '/private/tmp/v/bin/python', 'file_system_encoding': 'utf-8', 'has_venv': True, 'implementation': 'CPython', 'max_size': 9223372036854775807, 'original_executable': '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8', 'os': 'posix', 'path': ['/private/tmp/nox-extra-pythons/lib/python3.8/site-packages/virtualenv/discovery', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python38.zip', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8/lib-dynload', '/Users/cjolowicz/.local/lib/python3.8/site-packages', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8/site-packages'], 'platform': 'darwin', 'prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'real_prefix': None, 'stdout_encoding': 'utf-8', 'sysconfig': {'makefile_filename': '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8/config-3.8-darwin/Makefile'}, 'sysconfig_paths': {'data': '{base}', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'platlib': '{platbase}/lib/python{py_version_short}/site-packages', 'platstdlib': '{platbase}/lib/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'scripts': '{base}/bin', 'stdlib': '{installed_base}/lib/python{py_version_short}'}, 'sysconfig_vars': {'PYTHONFRAMEWORK': '', 'abiflags': '', 'base': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'installed_base': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'platbase': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'py_version_short': '3.8'}, 'system_executable': '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8', 'system_stdlib': '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8', 'system_stdlib_platform': '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8', 'version': '3.8.6 (default, Sep 26 2020, 17:16:54) \n[Clang 11.0.0 (clang-1100.0.20.17)]', 'version_info': VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)}) [DEBUG py_info:370]
139 discover exe from cache /Users/cjolowicz/.pyenv/versions/3.8.6 - exact False: PythonInfo({'architecture': 64, 'base_exec_prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'base_prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'distutils_install': {'data': '', 'headers': 'include/python3.8/UNKNOWN', 'platlib': 'lib/python3.8/site-packages', 'purelib': 'lib/python3.8/site-packages', 'scripts': 'bin'}, 'exec_prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'executable': '/private/tmp/v/bin/python3', 'file_system_encoding': 'utf-8', 'has_venv': True, 'implementation': 'CPython', 'max_size': 9223372036854775807, 'original_executable': '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8', 'os': 'posix', 'path': ['/private/tmp/nox-extra-pythons/lib/python3.8/site-packages/virtualenv/discovery', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python38.zip', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8/lib-dynload', '/Users/cjolowicz/.local/lib/python3.8/site-packages', '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8/site-packages'], 'platform': 'darwin', 'prefix': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'real_prefix': None, 'stdout_encoding': 'utf-8', 'sysconfig': {'makefile_filename': '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8/config-3.8-darwin/Makefile'}, 'sysconfig_paths': {'data': '{base}', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'platlib': '{platbase}/lib/python{py_version_short}/site-packages', 'platstdlib': '{platbase}/lib/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'scripts': '{base}/bin', 'stdlib': '{installed_base}/lib/python{py_version_short}'}, 'sysconfig_vars': {'PYTHONFRAMEWORK': '', 'abiflags': '', 'base': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'installed_base': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'platbase': '/Users/cjolowicz/.pyenv/versions/3.8.6', 'py_version_short': '3.8'}, 'system_executable': '/Users/cjolowicz/.pyenv/versions/3.8.6/bin/python3.8', 'system_stdlib': '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8', 'system_stdlib_platform': '/Users/cjolowicz/.pyenv/versions/3.8.6/lib/python3.8', 'version': '3.8.6 (default, Sep 26 2020, 17:16:54) \n[Clang 11.0.0 (clang-1100.0.20.17)]', 'version_info': VersionInfo(major=3, minor=8, micro=6, releaselevel='final', serial=0)}) [DEBUG py_info:370]
139 discover PATH[1]=/Users/cjolowicz/.pyenv/shims [DEBUG builtin:114]
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/shims/python3.7')
_get_via_file_cache: path_modified = 1623268689.907189
140 Attempting to acquire lock 4348973408 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/2e7a1013b824c3bb3dd95233f475c3d47396618eef5307097b95e27f44fb5849.lock [DEBUG filelock:270]
140 Lock 4348973408 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/2e7a1013b824c3bb3dd95233f475c3d47396618eef5307097b95e27f44fb5849.lock [INFO filelock:274]
141 got python info of /Users/cjolowicz/.pyenv/shims/python3.7 from /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/2e7a1013b824c3bb3dd95233f475c3d47396618eef5307097b95e27f44fb5849.json [DEBUG via_disk_folder:135]
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/shims/python3.7'
_get_via_file_cache: of_st_mtime = 1623268689.907189
141 Attempting to release lock 4348973408 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/2e7a1013b824c3bb3dd95233f475c3d47396618eef5307097b95e27f44fb5849.lock [DEBUG filelock:315]
141 Lock 4348973408 released on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/2e7a1013b824c3bb3dd95233f475c3d47396618eef5307097b95e27f44fb5849.lock [INFO filelock:318]
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
_get_via_file_cache:
_get_via_file_cache: path = PosixPath('/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7')
_get_via_file_cache: path_modified = 1599937426.8033798
142 Attempting to acquire lock 4348974224 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/37ed3ea709b941976d7b44855dcaa01f6a0303fca9396962b5d6911281420a6b.lock [DEBUG filelock:270]
142 Lock 4348974224 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/37ed3ea709b941976d7b44855dcaa01f6a0303fca9396962b5d6911281420a6b.lock [INFO filelock:274]
143 got python info of /Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7 from /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/37ed3ea709b941976d7b44855dcaa01f6a0303fca9396962b5d6911281420a6b.json [DEBUG via_disk_folder:135]
_get_via_file_cache: of_path = '/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7'
_get_via_file_cache: of_st_mtime = 1599937426.8033798
143 Attempting to release lock 4348974224 on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/37ed3ea709b941976d7b44855dcaa01f6a0303fca9396962b5d6911281420a6b.lock [DEBUG filelock:315]
143 Lock 4348974224 released on /Users/cjolowicz/Library/Application Support/virtualenv/py_info/1/37ed3ea709b941976d7b44855dcaa01f6a0303fca9396962b5d6911281420a6b.lock [INFO filelock:318]
_get_via_file_cache: return py_info.version_info = VersionInfo(major=3, minor=7, micro=9, releaselevel='final', serial=0)
143 proposed PathPythonInfo(spec=CPython3.7.9.final.0-64, system=/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7, exe=/Users/cjolowicz/.pyenv/shims/python3.7, platform=darwin, version='3.7.9 (default, Sep 12 2020, 21:03:01) \n[Clang 11.0.0 (clang-1100.0.20.17)]', encoding_fs_io=utf-8-UTF-8) [INFO builtin:69]
144 accepted PathPythonInfo(spec=CPython3.7.9.final.0-64, system=/Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7, exe=/Users/cjolowicz/.pyenv/shims/python3.7, platform=darwin, version='3.7.9 (default, Sep 12 2020, 21:03:01) \n[Clang 11.0.0 (clang-1100.0.20.17)]', encoding_fs_io=utf-8-UTF-8) [DEBUG builtin:71]
203 create virtual environment via CPython3Posix(dest=/private/tmp/vx, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
203 create folder /private/tmp/vx/bin [DEBUG _sync:25]
203 create folder /private/tmp/vx/lib/python3.7/site-packages [DEBUG _sync:25]
204 write /private/tmp/vx/pyvenv.cfg [DEBUG pyenv_cfg:34]
204 	home = /Users/cjolowicz/.pyenv/versions/3.7.9 [DEBUG pyenv_cfg:38]
204 	implementation = CPython [DEBUG pyenv_cfg:38]
204 	version_info = 3.7.9.final.0 [DEBUG pyenv_cfg:38]
204 	virtualenv = 20.4.7 [DEBUG pyenv_cfg:38]
204 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
204 	base-prefix = /Users/cjolowicz/.pyenv/versions/3.7.9 [DEBUG pyenv_cfg:38]
204 	base-exec-prefix = /Users/cjolowicz/.pyenv/versions/3.7.9 [DEBUG pyenv_cfg:38]
204 	base-executable = /Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7 [DEBUG pyenv_cfg:38]
205 symlink /Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7 to /private/tmp/vx/bin/python [DEBUG _sync:44]
210 create virtualenv import hook file /private/tmp/vx/lib/python3.7/site-packages/_virtualenv.pth [DEBUG api:95]
211 create /private/tmp/vx/lib/python3.7/site-packages/_virtualenv.py [DEBUG api:98]
212 ============================== target debug ============================== [DEBUG session:54]
214 debug via /private/tmp/vx/bin/python /private/tmp/v/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:224]
212 {
  "sys": {
    "executable": "/private/tmp/vx/bin/python",
    "_base_executable": null,
    "prefix": "/private/tmp/vx",
    "base_prefix": "/Users/cjolowicz/.pyenv/versions/3.7.9",
    "real_prefix": null,
    "exec_prefix": "/private/tmp/vx",
    "base_exec_prefix": "/Users/cjolowicz/.pyenv/versions/3.7.9",
    "path": [
      "/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python37.zip",
      "/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7",
      "/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/lib-dynload",
      "/private/tmp/vx/lib/python3.7/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "UTF-8"
  },
  "version": "3.7.9 (default, Sep 12 2020, 21:03:01) \n[Clang 11.0.0 (clang-1100.0.20.17)]",
  "makefile_filename": "/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/config-3.7m-darwin/Makefile",
  "os": "<module 'os' from '/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/os.py'>",
  "site": "<module 'site' from '/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/site.py'>",
  "datetime": "<module 'datetime' from '/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/datetime.py'>",
  "math": "<module 'math' from '/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/lib-dynload/math.cpython-37m-darwin.so'>",
  "json": "<module 'json' from '/Users/cjolowicz/.pyenv/versions/3.7.9/lib/python3.7/json/__init__.py'>"
} [DEBUG session:55]
466 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/cjolowicz/Library/Application Support/virtualenv) [INFO session:59]
469 got embed update of distribution setuptools from /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/embed/1/setuptools.json [DEBUG via_disk_folder:135]
469 got embed update of distribution pip from /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/embed/1/pip.json [DEBUG via_disk_folder:135]
469 got embed update of distribution wheel from /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/embed/1/wheel.json [DEBUG via_disk_folder:135]
474 got embed update of distribution setuptools from /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/embed/1/setuptools.json [DEBUG via_disk_folder:135]
475 got embed update of distribution pip from /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/embed/1/pip.json [DEBUG via_disk_folder:135]
475 got embed update of distribution wheel from /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/embed/1/wheel.json [DEBUG via_disk_folder:135]
476 install setuptools from wheel /private/tmp/v/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/setuptools-57.0.0-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
476 Attempting to acquire lock 4352231024 on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any.lock [DEBUG filelock:270]
476 install pip from wheel /private/tmp/v/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-21.1.2-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
477 install wheel from wheel /private/tmp/v/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/wheel-0.36.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
477 Attempting to acquire lock 4352361952 on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any.lock [DEBUG filelock:270]
477 Lock 4352231024 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any.lock [INFO filelock:274]
477 Attempting to acquire lock 4352363008 on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [DEBUG filelock:270]
478 Attempting to release lock 4352231024 on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any.lock [DEBUG filelock:315]
478 Lock 4352231024 released on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any.lock [INFO filelock:318]
478 Lock 4352361952 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any.lock [INFO filelock:274]
478 Attempting to release lock 4352361952 on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any.lock [DEBUG filelock:315]
478 copy /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any/setuptools-57.0.0.virtualenv to /private/tmp/vx/lib/python3.7/site-packages/setuptools-57.0.0.virtualenv [DEBUG _sync:52]
478 Lock 4352363008 acquired on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [INFO filelock:274]
478 Lock 4352361952 released on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any.lock [INFO filelock:318]
479 copy /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any/pip-21.1.2.virtualenv to /private/tmp/vx/lib/python3.7/site-packages/pip-21.1.2.virtualenv [DEBUG _sync:52]
479 Attempting to release lock 4352363008 on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [DEBUG filelock:315]
479 Lock 4352363008 released on /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [INFO filelock:318]
479 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any/wheel-0.36.2.dist-info to /private/tmp/vx/lib/python3.7/site-packages/wheel-0.36.2.dist-info [DEBUG _sync:52]
480 copy /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any/distutils-precedence.pth to /private/tmp/vx/lib/python3.7/site-packages/distutils-precedence.pth [DEBUG _sync:52]
480 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any/pip to /private/tmp/vx/lib/python3.7/site-packages/pip [DEBUG _sync:52]
481 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any/setuptools to /private/tmp/vx/lib/python3.7/site-packages/setuptools [DEBUG _sync:52]
494 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any/wheel to /private/tmp/vx/lib/python3.7/site-packages/wheel [DEBUG _sync:52]
525 copy /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any/wheel-0.36.2.virtualenv to /private/tmp/vx/lib/python3.7/site-packages/wheel-0.36.2.virtualenv [DEBUG _sync:52]
529 generated console scripts wheel3.7 wheel3 wheel wheel-3.7 [DEBUG base:48]
679 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any/pkg_resources to /private/tmp/vx/lib/python3.7/site-packages/pkg_resources [DEBUG _sync:52]
716 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any/setuptools-57.0.0.dist-info to /private/tmp/vx/lib/python3.7/site-packages/setuptools-57.0.0.dist-info [DEBUG _sync:52]
728 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-57.0.0-py3-none-any/_distutils_hack to /private/tmp/vx/lib/python3.7/site-packages/_distutils_hack [DEBUG _sync:52]
737 generated console scripts  [DEBUG base:48]
964 copy directory /Users/cjolowicz/Library/Application Support/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-21.1.2-py3-none-any/pip-21.1.2.dist-info to /private/tmp/vx/lib/python3.7/site-packages/pip-21.1.2.dist-info [DEBUG _sync:52]
971 generated console scripts pip3.7 pip pip-3.7 pip3 [DEBUG base:48]
971 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
975 write /private/tmp/vx/pyvenv.cfg [DEBUG pyenv_cfg:34]
975 	home = /Users/cjolowicz/.pyenv/versions/3.7.9 [DEBUG pyenv_cfg:38]
975 	implementation = CPython [DEBUG pyenv_cfg:38]
975 	version_info = 3.7.9.final.0 [DEBUG pyenv_cfg:38]
975 	virtualenv = 20.4.7 [DEBUG pyenv_cfg:38]
975 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
975 	base-prefix = /Users/cjolowicz/.pyenv/versions/3.7.9 [DEBUG pyenv_cfg:38]
975 	base-exec-prefix = /Users/cjolowicz/.pyenv/versions/3.7.9 [DEBUG pyenv_cfg:38]
975 	base-executable = /Users/cjolowicz/.pyenv/versions/3.7.9/bin/python3.7 [DEBUG pyenv_cfg:38]
976 created virtual environment CPython3.7.9.final.0-64 in 867ms
  creator CPython3Posix(dest=/private/tmp/vx, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/cjolowicz/Library/Application Support/virtualenv)
    added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:19]
@cjolowicz cjolowicz added the bug label Jun 9, 2021
@cjolowicz cjolowicz mentioned this issue Jun 9, 2021
Closed
@cjolowicz cjolowicz changed the title Cache invalidation broken on Mac with pyenv? Cache invalidation broken with pyenv? Jun 10, 2021
@cjolowicz cjolowicz mentioned this issue Jun 10, 2021
Closed
11 tasks
@cjolowicz
Copy link
Contributor Author

I've sent a PR to the pyenv project that would fix this for the future, at least. See pyenv/pyenv#1981

@brechtm
Copy link

brechtm commented Jun 19, 2021

To get around this temporarily, find out where virtualenv's cache is located and delete it:

virtualenv --help | grep cache

@gaborbernat
Copy link
Contributor

gaborbernat commented Aug 6, 2021 •
edited
Loading

I'd be happy to add a flag that disables the cache behaviour and pyenv users can set that on their profile (note there are a user-level virtualenv.ini where you can change the behaviour of virtualenv). PR in this sense are welcome.

@fschulze fschulze mentioned this issue Aug 6, 2021
Closed
@gaborbernat gaborbernat mentioned this issue Aug 16, 2021
Closed
@cjolowicz cjolowicz mentioned this issue Dec 27, 2021
Closed
@cjolowicz cjolowicz mentioned this issue Feb 24, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug help-wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@brechtm @cjolowicz @gaborbernat