FakePathlibModule shows real files on Python 3.11+ (works on 3.10)

[evgenii-moriakhin]

With Python 3.11 and 3.12 the fake pathlib no longer stays inside the fake filesystem.

Calling Path('/').iterdir() returns the real files from the container.

On Python 3.10 the same code returns an empty list, which is what I expect.

How to reproduce:

Python 3.10 – works

docker run -it --rm python:3.10-alpine sh -c "
pip install -q pyfakefs && \
python - <<'PY'
from pyfakefs.fake_filesystem import FakeFilesystem
from pyfakefs.fake_pathlib import FakePathlibModule
pathlib = FakePathlibModule(FakeFilesystem())
print(list(pathlib.Path('/').iterdir()))   # expected []
PY
"

Python 3.12 – broken

docker run -it --rm python:3.12-alpine sh -c "
pip install -q pyfakefs && \
python - <<'PY'
from pyfakefs.fake_filesystem import FakeFilesystem
from pyfakefs.fake_pathlib import FakePathlibModule
pathlib = FakePathlibModule(FakeFilesystem())
print(list(pathlib.Path('/').iterdir()))   # shows real /bin, /usr, ...
PY
"

Why I think it breaks:
pathlib versions in newer Pythons use real os module so the fake filesystem is bypassed.

[evgenii-moriakhin]

as workaround:

import os
import sys

from pyfakefs.fake_filesystem import FakeFilesystem
from pyfakefs.fake_os import FakeOsModule
from pyfakefs.fake_pathlib import FakePathlibModule

filesystem = FakeFilesystem()
fake_os = FakeOsModule(filesystem)

del sys.modules["pathlib"]

sys.modules["os"] = fake_os

for module in dict(sys.modules):
    if module.startswith("pyfakefs."):
        del sys.modules[module]

from pyfakefs.fake_pathlib import FakePathlibModule

pathlib = FakePathlibModule(filesystem)

sys.modules["os"] = os

print(list(pathlib.Path("/").iterdir()))
pathlib.Path("/foo").touch()
print(list(pathlib.Path("/").iterdir()))

[mrbean-bremen]

Thanks, I will have a look!
It is true that pathlib uses os in Python 3.11+, and we patch that if using pyfakefs via the Patcher (e.g. using the fs fixture, or fake_filesystem_unittest.TestCase, or the patchfs decorator).
I've probably never tested instantiating FakePathlibModule directly, and I can see why it won't work. Maybe it should be changed to automatically create a FakeOsModule if non exists yet (basically what you are doing in your workaround), or something like this - I have to think about it...

[evgenii-moriakhin]

My use case for creating a module directly involves complex integration tests that “simulate” several physical nodes, each with its own file system.
Because the application is multithreaded, patching or mocking would be a bad idea, so the code under test has been reorganized to allow direct injection of file-system-related libraries (open, os, pathlib, etc.) into the tested code—for example, through calls such as get_os_lib().

The injection itself is done by installing a fake module via a ContextVar (or by returning the default module if no fake one has been set).

I realize my scenario is rather complicated, but it explains why I came across this need

[mrbean-bremen]

Sure, this is a valid use case - as I said, I have neglected this so far, probably need at least some basic tests for this.

[mrbean-bremen]

@evgenii-moriakhin - shall be fixed in main branch, please check! I cannot exclude any unwanted side-effect, as this has been tested only marginally, though.