Invalid pathlib.Path prepended to sys.path inside virtualenv using outside IPython

[reillysiemens]

Software Versions Tested

🐍 Python	3.9.7, 3.10.0rc2
✔️ Passing IPython	7.27.0
❌ Failing IPython	7.28.0

Issue Description

With fresh installations of the latest Python and IPython I cannot use an IPython installed outside of a virtualenv to import modules inside that virtualenv.

I assume from the UserWarning

Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.

that this workflow is supported on a best-effort basis, if at all. I understand if this isn't a high priority issue. That said, it is handy to not install IPython into each virtualenv I use. Historically it has worked quite well in this regard.

As far as I can determine, the cause of the issue is an unexpected pathlib.Path instance prepended to sys.path.

In [1]: import sys

In [2]: sys.path[0]
Out[2]: PosixPath('/tmp/venv/lib/python3.9/site-packages')

Near as I can tell from the documentation and cursory searching, sys.path should only contain str instances.

I don't know the IPython codebase well enough to say for sure, but from searching through issues and PRs it seems like this might have been an unanticipated result of #12548 and its 7.x backport #13146.

The issue does not seem to be reproducible in IPython 7.27.0, so that would seem to support this claim as the aforementioned backport merged a few days ago prior to the release of IPython 7.28.0.

If that is the case, maybe it would suffice to replace

ipython/IPython/core/interactiveshell.py

Line 946 in c31fed6

sys.path.insert(0, virtual_env)

with something like

sys.path.insert(0, os.fspath(virtual_env))

Steps to Reproduce

1. Install IPython outside a virtualenv.

   pip3.9 install ipython

2. Create a virtualenv.

   python3.9 -m venv venv

3. Source the virtualenv.

   source ./venv/bin/activate

4. Install any new package.

   pip install rich

5. Import that package from Python.

   python -c 'import rich'

6. Fail to import the same package from IPython.

   ipython -c 'import rich'
   /home/tucker/.pyenv/versions/3.9.7/lib/python3.9/site-packages/IPython/core/interactiveshell.py:934: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
     warn("Attempting to work in a virtualenv. If you encounter problems, please "
   ---------------------------------------------------------------------------
   ModuleNotFoundError                       Traceback (most recent call last)
   <ipython-input-1-5f03bc13fcf8> in <module>
   ----> 1 import rich
   
   ModuleNotFoundError: No module named 'rich'

sys.path Entry Comparisons

In case it helps, here are diffs of my sys.path inside and outside of a virtualenv using Python and IPython.

Python outside vs. inside a venv

  '/home/tucker/.pyenv/versions/3.9.7/lib/python39.zip',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9/lib-dynload',
- '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9/site-packages']
+ '/tmp/venv/lib/python3.9/site-packages']

IPython outside vs. inside a venv

-['/home/tucker/.pyenv/versions/3.9.7/bin',
+[PosixPath('/tmp/venv/lib/python3.9/site-packages'),
+ '/home/tucker/.pyenv/versions/3.9.7/bin',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python39.zip',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9/lib-dynload',
  '',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9/site-packages',
  '/home/tucker/.pyenv/versions/3.9.7/lib/python3.9/site-packages/IPython/extensions',
  '/home/tucker/.ipython']

[sivel]

I ran across another issue, as well as this one. I've posted a potential fix at #13177 (comment) that I believe should resolve both issues.

[reillysiemens]

@sivel: You mentioned in #13189 that this issue is already addressed elsewhere. Where is that?

[sivel]

My apologies, I should have provided that.

There were a few fixes that went in, but specifically on platforms other than windows, this resolved it:

62e6c27

The code in the master branch has been modified slightly since then, but still contains the cast to str:

ipython/IPython/core/interactiveshell.py

Line 949 in 1eeb3a4

virtual_env = str(virtual_env_path).format(*p_ver)

There is a PR open to address the windows path at #13170