MSYS2: RecursionError: maximum recursion depth exceeded

[turutcrane]

Issue description

I tryied pipenv==2023.5.19 to resolve #5446.
But RecursionError: maximum recursion depth exceeded occured.

Expected result

No RecursionError should occur.

Actual result

Copied log from #5446 (comment) in later section

Steps to replicate

PYTHONUSERBASE=/home/turu/py-usersite
PIPENV_VENV_IN_PROJECT=1
WORKON_HOME=/home/turu/py-venvs

Used command

$ mkdir pipenv-test
$ cd pipenv-test
$ pacman -S mingw64/mingw-w64-x86_64-python-pip
$ pip install --user pipenv==2023.5.19
$ pipenv install
$ pipenv shell

Some Details

Log show infiline call loop start from

packages/pipenv/vendor/pythonfinder/models/mixins.py", line 304, in find_python_version
    results = sorted(matching_pythons, key=lambda r: (r[1], r[0]), reverse=True)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-```

The line above calls the following routine:

packages/pipenv/vendor/pydantic/main.py", line 909, in __eq__
    return self.dict() == other.dict()

So, I add the following method to class PathEntry(BaseModel) in vendor/pythonfinder/models/mixins.py.

    def __eq__(self, other) -> bool:
        return self.path.as_posix() == other.path.as_posix()

Then, RecursionError disappeared.

A Strange things is that I change vendor/pythonfinder/models/mixins.py", line 304 to

    results = sorted(matching_pythons, key=lambda r: (r[0]), reverse=True)

Then also RecursionError disappeared. (I know this change logic)

log copied from #5446 (comment)

15:28$ pip install --user pipenv==2023.5.19
Collecting pipenv==2023.5.19
  Using cached pipenv-2023.5.19-py3-none-any.whl (2.9 MB)
Requirement already satisfied: certifi in d:/diskd/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages (from pipenv==2023.5.19) (2023.5.7)
Requirement already satisfied: setuptools>=67.0.0 in d:/diskd/develop/msys64/mingw64/lib/python3.10/site-packages (from pipenv==2023.5.19) (67.7.2)
Requirement already satisfied: virtualenv-clone>=0.2.5 in d:/diskd/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages (from pipenv==2023.5.19) (0.5.7)
Requirement already satisfied: virtualenv>=20.17.1 in d:/diskd/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages (from pipenv==2023.5.19) (20.23.0)
Requirement already satisfied: distlib<1,>=0.3.6 in d:/diskd/develop/msys64/mingw64/lib/python3.10/site-packages (from virtualenv>=20.17.1->pipenv==2023.5.19) (0.3.6)
Requirement already satisfied: filelock<4,>=3.11 in d:/diskd/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages (from virtualenv>=20.17.1->pipenv==2023.5.19) (3.12.0)
Requirement already satisfied: platformdirs<4,>=3.2 in d:/diskd/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages (from virtualenv>=20.17.1->pipenv==2023.5.19) (3.5.1)
Installing collected packages: pipenv
Successfully installed pipenv-2023.5.19

16:06$ pipenv install
Creating a virtualenv for this project...
Pipfile: D:/DiskD/develop/pipenv-test/Pipfile
Using default python from D:/DiskD/develop/msys64/mingw64/bin/python.exe (3.10.11) to create virtualenv...
[ ===] Creating virtual environment...created virtual environment CPython3.10.11.final.0-64 in 337ms
  creator CPython3Windows(dest=D:/DiskD/develop/pipenv-test/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:/Users/turu/AppData/Local/pypa/virtualenv)
    added seed packages: pip==23.1.2, setuptools==67.7.2, wheel==0.40.0
  activators BashActivator,BatchActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

Successfully created virtual environment!
Virtualenv location: D:/DiskD/develop/pipenv-test/.venv
Creating a Pipfile for this project...
Creating a virtualenv for this project...
Pipfile: D:/DiskD/develop/pipenv-test/Pipfile
Using default python from D:/DiskD/develop/msys64/mingw64/bin/python.exe (3.10.11) to create virtualenv...
[ ===] Creating virtual environment...created virtual environment CPython3.10.11.final.0-64 in 387ms
  creator CPython3Windows(dest=D:/DiskD/develop/pipenv-test/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:/Users/turu/AppData/Local/pypa/virtualenv)
    added seed packages: pip==23.1.2, setuptools==67.7.2, wheel==0.40.0
  activators BashActivator,BatchActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

Successfully created virtual environment!

Virtualenv location: D:/DiskD/develop/pipenv-test/.venv
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Updated Pipfile.lock (fedbd2ab7afd84cf16f128af0619749267b62277b4cb6989ef16d4bef6e4eef2)!
Installing dependencies from Pipfile.lock (e4eef2)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

16:19$ pipenv shell
Traceback (most recent call last):
  File "D:/DiskD/develop/msys64/mingw64/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:/DiskD/develop/msys64/mingw64/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/bin/pipenv.exe/__main__.py", line 7, in <module>
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/cli/command.py", line 429, in shell
    do_shell(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/routines/shell.py", line 14, in do_shell
    ensure_project(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/utils/project.py", line 32, in ensure_project
    ensure_virtualenv(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/utils/virtualenv.py", line 148, in ensure_virtualenv
    python = ensure_python(project, python=python)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/utils/virtualenv.py", line 242, in ensure_python
    path_to_python = find_a_system_python(python)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/utils/virtualenv.py", line 359, in find_a_system_python
    python_entry = find_python(finder, line)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/utils/shell.py", line 259, in find_python
    result = finder.find_python_version(line)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 174, in find_python_version
    return self.system_path.find_python_version(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 471, in find_python_version
    ver = next(iter(self.get_pythons(sub_finder)), None)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 405, in get_pythons
    pythons = [entry for entry in self._get_all_pythons(finder)]
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 405, in <listcomp>
    pythons = [entry for entry in self._get_all_pythons(finder)]
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 397, in _get_all_pythons
    for python in self._filter_paths(finder):
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 390, in _filter_paths
    python_versions = finder(path)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 449, in sub_finder
    return obj.find_python_version(major, minor, patch, pre, dev, arch, name)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pythonfinder/models/mixins.py", line 304, in find_python_version
    results = sorted(matching_pythons, key=lambda r: (r[1], r[0]), reverse=True)
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 909, in __eq__
    return self.dict() == other.dict()
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 449, in dict
    return dict(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 868, in _iter
    v = self._get_value(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 761, in _get_value
    return {
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 762, in <dictcomp>
    k_: cls._get_value(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 743, in _get_value
    v_dict = v.dict(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 449, in dict
    return dict(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 868, in _iter

   loop ,,,,

   File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 449, in dict
    return dict(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 868, in _iter
    v = self._get_value(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 761, in _get_value
    return {
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 762, in <dictcomp>
    k_: cls._get_value(
  File "D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv/vendor/pydantic/main.py", line 741, in _get_value
    if isinstance(v, BaseModel):
RecursionError: maximum recursion depth exceeded

---

$ pipenv --support

Pipenv version: '2023.5.19'

Pipenv location: 'D:/DiskD/develop/msys64/home/turu/py-usersite/lib/python3.10-mingw_x86_64/site-packages/pipenv'

Python location: 'D:/DiskD/develop/msys64/mingw64/bin/python.exe'

OS Name: 'nt'

User pip version: '23.1.2'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.10.11',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '10',
 'platform_system': 'Windows',
 'platform_version': '10.0.22621',
 'python_full_version': '3.10.11',
 'python_version': '3.10',
 'sys_platform': 'win32'}

System environment variables:

• ACLOCAL_PATH
• ALLUSERSPROFILE
• APPDATA
• COMMONPROGRAMFILES
• COMPUTERNAME
• COMSPEC
• CONFIG_SITE
• COMMONPROGRAMFILES(X86)
• COMMONPROGRAMW6432
• DRIVERDATA
• GOPATH
• HOME
• HOMEDRIVE
• HOMEPATH
• HOSTNAME
• INFOPATH
• LANG
• LOCALAPPDATA
• LOGONSERVER
• MANPATH
• MINGW_CHOST
• MINGW_PACKAGE_PREFIX
• MINGW_PREFIX
• MSYSTEM
• MSYSTEM_CARCH
• MSYSTEM_CHOST
• MSYSTEM_PREFIX
• NUMBER_OF_PROCESSORS
• OLDPWD
• ORIGINAL_PATH
• ORIGINAL_TEMP
• ORIGINAL_TMP
• OS
• ONEDRIVE
• ONEDRIVECONSUMER
• PATH
• PATHEXT
• PIPENV_VENV_IN_PROJECT
• PKG_CONFIG_PATH
• PKG_CONFIG_SYSTEM_INCLUDE_PATH
• PKG_CONFIG_SYSTEM_LIBRARY_PATH
• POWERSHELL_DISTRIBUTION_CHANNEL
• PRINTER
• PROCESSOR_ARCHITECTURE
• PROCESSOR_IDENTIFIER
• PROCESSOR_LEVEL
• PROCESSOR_REVISION
• PROGRAMFILES
• PROMPT
• PS1
• PSMODULEPATH
• PUBLIC
• PWD
• PYTHONUSERBASE
• PROGRAMDATA
• PROGRAMFILES(X86)
• PROGRAMW6432
• SHELL
• SHLVL
• SSH_AGENT_PID
• SSH_AUTH_SOCK
• SYSTEMDRIVE
• SYSTEMROOT
• TEMP
• TERM
• TMP
• USER
• USERDOMAIN
• USERDOMAIN_ROAMINGPROFILE
• USERNAME
• USERPROFILE
• WINDIR
• WORKON_HOME
• WSLENV
• WT_PROFILE_ID
• WT_SESSION
• XDG_DATA_DIRS
• ZES_ENABLE_SYSMAN
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PIP_PYTHON_PATH
• PYTHONDONTWRITEBYTECODE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

• PIPENV_VENV_IN_PROJECT: 1

Debug–specific environment variables:

• PATH: D:\DiskD\develop\msys64\home\turu\py-usersite\bin;C:\Program Files\Amazon\SessionManagerPlugin\bin;C:\Program Files\Amazon\AWSCLIV2;D:\DiskD\develop\msys64\home\turu\bin;D:\DiskD\develop\msys64\mingw64\bin;D:\DiskD\develop\msys64\usr\local\bin;D:\DiskD\develop\msys64\usr\bin;C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\DiskD\develop\msys64\usr\bin\site_perl;D:\DiskD\develop\msys64\usr\bin\vendor_perl;D:\DiskD\develop\msys64\usr\bin\core_perl;D:\DiskD\develop\msys64\mingw64\bin\
• SHELL: D:\DiskD\develop\msys64\usr\bin\bash.exe
• LANG: ja_JP.UTF-8
• PWD: D:/DiskD/develop/codecommit/flet9

---

Contents of Pipfile ('D:/DiskD/develop/codecommit/flet9/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.10"

Contents of Pipfile.lock ('D:/DiskD/develop/codecommit/flet9/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "fedbd2ab7afd84cf16f128af0619749267b62277b4cb6989ef16d4bef6e4eef2"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.10"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

[matteius]

I think the idea of adding the equality check on the PathEntry makes sense to me @turutcrane -- do you have capacity to open a PR against pythonfinder for this?

[turutcrane]

I have opened PR sarugaku/pythonfinder#141.

[AdriannaBeck]

Hello! I believe I am facing a similar issue. I see that the fix has been merged - do you know when a new release may become available that includes this fix? Thanks in advance!

[matteius]

I'm waiting on a couple more PRs to get reviewed before I cut the next release, hopefully can happen over the weekend. I wouldn't want to release without including this one at a minimum: #5712

[matteius]

pipenv==2023.6.2 has been released