Changing cwd in a Linux mocked filesystem in a Windows environment causes confusion

[giladreti]

Describe the bug
When changing the current working directory while using the patched filesystem that mocks a Linux filesystem, the fake filesystem gets confused when trying to access Windows absolute paths, since absolute paths in Windows are not prefixed with a '/' while in Linux they are. The fake filesystem tries to resolve the path as a relative path in the current working directory and fails.

Traceback (most recent call last):
  File "C:/Users/gilad/Documents/my_module/bug.py", line 19, in <module>
    bug()
  File "C:/Users/gilad/Documents/my_module/bug.py", line 16, in bug
    open(__file__)
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 4449, in open
    return fake_open(file, mode, buffering, encoding, errors,
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 4935, in __call__
    return self.call(*args, **kwargs)
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 4983, in call
    file_object = self._init_file_object(file_object,
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 5037, in _init_file_object
    self.filesystem.raise_os_error(errno.ENOENT, file_path)
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 960, in raise_os_error
    raise OSError(errno, message, filename)
FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: 'C:/Users/gilad/Documents/my_module/bug.py'

How To Reproduce

My project tree structure:

.
├── __pycache__
│   └── test.cpython-38-pytest-6.0.2.pyc
└── bug.py

1 directory, 2 files

My test file:

import os
from pathlib import Path

from pyfakefs.fake_filesystem_unittest import Patcher


def bug():
    with Patcher() as patcher:
        filesystem = patcher.fs
        filesystem.is_windows_fs = filesystem.is_macos = False

        filesystem.add_real_file(__file__)

        open(__file__) # works
        os.chdir(Path.home())
        open(__file__) # fails


bug()

Your enviroment
Please run the following and paste the output.

python -c "import platform; print(platform.platform())"
python -c "import sys; print('Python', sys.version)"
python -c "from pyfakefs.fake_filesystem import __version__; print('pyfakefs', __version__)"

Output:

C:\Users\gilad\Documents\my_module>python -c "import platform; print(platform.platform())"
Windows-10-10.0.19041-SP0

C:\Users\gilad\Documents\my_module>python -c "import sys; print('Python', sys.version)"
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]

C:\Users\gilad\Documents\my_module>python -c "from pyfakefs.fake_filesystem import __version__; print('pyfakefs', __version__)"
pyfakefs 4.1.0

[mrbean-bremen]

Thanks for the report!

Some background: Setting the filesystem OS in the tests is something we haven't paid too much attention, as it seemed not to be used much outside of pyfakefs itself. We use this quite a lot for our own tests, and I know that there are a few shortcomings. Some of the faked code (mainly pathlib) relies on the real implementation, and therefore switching the OS may not work there correctly, though that is something that I wanted to address at some time anyway.

I will have a closer look at the issue later, maybe there is an easy fix.

[giladreti]

I think that any path that starts with a windows drive should be resolved as an absolute path only as a fallback (or you can force the developer to add a leading backslash to indicate it is indeed an absolute path) since it will cause resolving relative paths as absolute. There may be a better solution, I couldn't think about one now.
With the current changes, the following code will break:

import os
from pathlib import Path

from pyfakefs.fake_filesystem import OSType
from pyfakefs.fake_filesystem_unittest import Patcher

def bug():

    with Patcher() as patcher:

        filesystem = patcher.fs
        filesystem.os = OSType.LINUX

        folder = Path('/test')
        file = folder / 'C:/testfile'

        file.parent.mkdir(parents=True)
        file.touch()

        os.chdir(folder)
        open(file.relative_to(folder))  # fail - FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: PosixPath('C:/testfile')



bug()

Since it will resolve C:/testfile as /C:/testfile and not as an absolute path.
I am starting to think that this was not a bug in the first place but more of a design constraint...

stacktrace:

Traceback (most recent call last):
  File "C:/Users/gilad/Documents/my_module/bug.py", line 25, in <module>
    bug()
  File "C:/Users/gilad/Documents/my_module/bug.py", line 22, in bug
    open(file.relative_to(folder))
  File "c:\users\gilad\.virtualenvs\my_module-q9ys3txz\src\pyfakefs\pyfakefs\fake_filesystem.py", line 4543, in open
    return fake_open(file, mode, buffering, encoding, errors,
  File "c:\users\gilad\.virtualenvs\my_module-q9ys3txz\src\pyfakefs\pyfakefs\fake_filesystem.py", line 5131, in __call__
    return self.call(*args, **kwargs)
  File "c:\users\gilad\.virtualenvs\my_module-q9ys3txz\src\pyfakefs\pyfakefs\fake_filesystem.py", line 5187, in call
    file_object = self._init_file_object(file_object,
  File "c:\users\gilad\.virtualenvs\my_module-q9ys3txz\src\pyfakefs\pyfakefs\fake_filesystem.py", line 5242, in _init_file_object
    self.filesystem.raise_os_error(errno.ENOENT, file_path)
  File "c:\users\gilad\.virtualenvs\my_module-q9ys3txz\src\pyfakefs\pyfakefs\fake_filesystem.py", line 1000, in raise_os_error
    raise OSError(errno, message, filename)
FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: PosixPath('C:/testfile')

[mrbean-bremen]

The combination of real files (added by add_real_xxx or from __file) with another file system is a bit of a problem, especially with drive letters. I don't have a good solution, and any solution will be make some sort of compromise.
As for your example--you are right, this is a case that now doesn't work because of that compromise made for real files to work, so maybe it wasn't such a good idea. I will have a closer look tonight.

[mrbean-bremen]

Ok, I made the compromise a bit less problematic, while still retaining the ability to handle real files. It will not solve all cases, but hopefully most.
Waiting for more issues :)