diff --git a/CHANGES.md b/CHANGES.md index 2326388e..5fb133c4 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,26 +1,31 @@ # pyfakefs Release Notes The release versions are PyPi releases. -## Version 3.2 (as yet unreleased) +## Version 3.2 #### New Features - * `io.open`, `os.open`: support for `errors` argument - * Added new methods to `fake_filesystem.FakeFilesystem` that make real files - and directories appear within the fake file system: - `add_real_file()`, `add_real_directory()` and `add_real_paths()`. - File contents are read from the real file system only when needed ([#170](../../issues/170)). - * Added the CHANGES.md release notes to the release manifest + * The `errors` argument is supported for `io.open()` and `os.open()` + * New methods `add_real_file()`, `add_real_directory()` and `add_real_paths()` + make real files and directories appear within the fake file system. + File contents are read from the real file system only as needed ([#170](../../issues/170)). + See `example_test.py` for a usage example. + * Deprecated `TestCase.copyRealFile()` in favor of `add_real_file()`. + `copyRealFile()` remains only for backward compatability. Also, some + less-popular argument combinations have been disallowed. + * Added this file you are reading, `CHANGES.md`, to the release manifest #### Infrastructure - * `mox3` is no longer required - the relevant part has been integrated into pyfakefs ([#182](../../issues/182)) + * The `mox3` package is no longer a prerequisite--the portion required by pyfakefs + has been integrated into pyfakefs ([#182](../../issues/182)) #### Fixes - * Corrected handling of byte/unicode paths in several functions ([#187](../../issues/187)) - * `FakeShutilModule.rmtree` failed for directory ending with path separator ([#177](../../issues/177)) - * Case incorrectly handled for added Windows drives + * Corrected the handling of byte/unicode paths in several functions ([#187](../../issues/187)) + * `FakeShutilModule.rmtree()` failed for directories ending with path separator ([#177](../../issues/177)) + * Case was incorrectly handled for added Windows drives * `pathlib.glob()` incorrectly handled case under MacOS ([#167](../../issues/167)) * tox support was broken ([#163](../../issues/163)) - * Rename that only changes case was not possible under Windows ([#160](../../issues/160)) + * On Windows it was not possible to rename a file when only the case of the file + name changed ([#160](../../issues/160)) ## [Version 3.1](https://pypi.python.org/pypi/pyfakefs/3.1) @@ -37,20 +42,20 @@ The release versions are PyPi releases. ## [Version 3.0](https://pypi.python.org/pypi/pyfakefs/3.0) #### New Features - * support for path-like objects as arguments in fake `os` + * Support for path-like objects as arguments in fake `os` and `os.path` modules (Python >= 3.6) - * some changes to make pyfakefs work with Python 3.6 - * added fake `pathlib` module (Python >= 3.4) ([#29](../../issues/29)) - * support for `os.replace` (Python >= 3.3) + * Some changes to make pyfakefs work with Python 3.6 + * Added fake `pathlib` module (Python >= 3.4) ([#29](../../issues/29)) + * Support for `os.replace` (Python >= 3.3) * `os.access`, `os.chmod`, `os.chown`, `os.stat`, `os.utime`: support for `follow_symlinks` argument (Python >= 3.3) - * support for `os.scandir` (Python >= 3.5) ([#119](../../issues/119)) - * option to not fake modules named `path` ([#53](../../issues/53)) + * Support for `os.scandir` (Python >= 3.5) ([#119](../../issues/119)) + * Option to not fake modules named `path` ([#53](../../issues/53)) * `glob.glob`, `glob.iglob`: support for `recursive` argument (Python >= 3.5) ([#116](../../issues/116)) - * support for `glob.iglob` ([#59](../../issues/59)) + * Support for `glob.iglob` ([#59](../../issues/59)) #### Infrastructure - * added [auto-generated documentation](http://jmcgeheeiv.github.io/pyfakefs/) + * Added [auto-generated documentation](http://jmcgeheeiv.github.io/pyfakefs/) #### Fixes * `shutil.move` incorrectly moves directories ([#145](../../issues/145)) @@ -64,57 +69,55 @@ The release versions are PyPi releases. #### New Features * `io.open`, `os.open`: support for `encoding` argument ([#120](../../issues/120)) * `os.makedirs`: support for `exist_ok` argument (Python >= 3.2) ([#98](../../issues/98)) - * support for fake `io.open()` ([#70](../../issues/70)) - * support for mount points ([#25](../../issues/25)) - * support for hard links ([#75](../../issues/75)) - * support for float times (mtime, ctime) + * Support for fake `io.open()` ([#70](../../issues/70)) + * Support for mount points ([#25](../../issues/25)) + * Support for hard links ([#75](../../issues/75)) + * Support for float times (mtime, ctime) * Windows support: * support for alternative path separator * support for case-insensitive filesystems ([#69](../../issues/69)) * support for drive letters and UNC paths - * support for filesystem size ([#86](../../issues/86)) + * Support for filesystem size ([#86](../../issues/86)) * `shutil.rmtree`: support for `ignore_errors` and `onerror` arguments ([#72](../../issues/72)) - * support for `os.fsync()` and `os.fdatasync()` ([#73](../../issues/73)) + * Support for `os.fsync()` and `os.fdatasync()` ([#73](../../issues/73)) * `os.walk`: Support for `followlinks` argument #### Fixes * `shutil` functions like `make_archive` do not work with pyfakefs ([#104](../../issues/104)) - * file permissions on deletion not correctly handled ([#27](../../issues/27)) + * File permissions on deletion not correctly handled ([#27](../../issues/27)) * `shutil.copy` error with bytes contents ([#105](../../issues/105)) * mtime and ctime not updated on content changes ## [Version 2.7](https://pypi.python.org/pypi/pyfakefs/2.7) #### Infrastructure - * moved repository from GoogleCode to GitHub, merging 3 projects - * added continuous integration testing with Travis CI - * added usage documentation in project wiki - * better support for pypi releases + * Moved repository from GoogleCode to GitHub, merging 3 projects + * Added continuous integration testing with Travis CI + * Added usage documentation in project wiki + * Better support for pypi releases #### New Features - * added direct unit test support in `fake_filesystem_unittest` + * Added direct unit test support in `fake_filesystem_unittest` (transparently patches all calls to faked implementations) - * added support for doctests - * added support for cygwin - * better support for Python 3 + * Added support for doctests + * Added support for cygwin + * Better support for Python 3 #### Fixes * `os.utime` fails to traverse symlinks ([#49](../../issues/49)) * `chown` incorrectly accepts non-integer uid/gid arguments ([#30](../../issues/30)) * Reading from fake block devices doesn't work ([#24](../../issues/24)) * `fake_tempfile` is using `AddOpenFile` incorrectly ([#23](../../issues/23)) - * incorrect behavior of `relpath`, `abspath` and `normpath` on Windows. - * cygwin wasn't treated as Windows ([#37](../../issues/37)) + * Incorrect behavior of `relpath`, `abspath` and `normpath` on Windows. + * Cygwin wasn't treated as Windows ([#37](../../issues/37)) * Python 3 `open` in binary mode not working ([#32](../../issues/32)) * `os.remove` doesn't work with relative paths ([#31](../../issues/31)) * `mkstemp` returns no valid file descriptor ([#19](../../issues/19)) * `open` methods lack `IOError` for prohibited operations ([#18](../../issues/18)) - * incorrectly resolved relative path ([#3](../../issues/3)) + * Incorrectly resolved relative path ([#3](../../issues/3)) * `FakeFileOpen` keyword args do not match the `__builtin__` equivalents ([#5](../../issues/5)) - * relative paths not supported ([#16](../../issues/16), [#17](../../issues/17))) + * Relative paths not supported ([#16](../../issues/16), [#17](../../issues/17))) ## Older Versions -As there have been three different projects that have been merged together -for release 2.7, no older release notes are given. -The following versions are still available in PyPi: +There are no release notes for releases 2.6 and below. The following versions are still available on PyPi: * [1.1](https://pypi.python.org/pypi/pyfakefs/1.1), [1.2](https://pypi.python.org/pypi/pyfakefs/1.2), [2.0](https://pypi.python.org/pypi/pyfakefs/2.0), [2.1](https://pypi.python.org/pypi/pyfakefs/2.1), [2.2](https://pypi.python.org/pypi/pyfakefs/2.2), [2.3](https://pypi.python.org/pypi/pyfakefs/2.3) and [2.4](https://pypi.python.org/pypi/pyfakefs/2.4) diff --git a/example_test.py b/example_test.py index a3aa47e4..e28b54d1 100644 --- a/example_test.py +++ b/example_test.py @@ -27,11 +27,10 @@ and allows you to specify the contents or the size of the file. """ +import io import os import sys -from pyfakefs.fake_filesystem_unittest import REAL_OPEN - if sys.version_info < (2, 7): import unittest2 as unittest else: @@ -65,6 +64,11 @@ def setUp(self): :py:class:`pyfakefs.mox3_stubout.StubOutForTesting`. Use this if you need to define additional stubs. """ + + # This is before setUpPyfakefs(), so still using the real file system + with io.open(__file__, 'rb') as f: + self.real_contents = f.read() + self.setUpPyfakefs() def tearDown(self): @@ -146,12 +150,11 @@ def test_scandir(self): def test_real_file_access(self): """Test `example.file_contents()` for a real file after adding it using `add_real_file()`.""" - real_file = __file__ - with REAL_OPEN(real_file, 'rb') as f: - real_contents = f.read() - self.assertRaises(IOError, example.file_contents, real_file) - self.fs.add_real_file(real_file) - self.assertEqual(example.file_contents(real_file), real_contents) + filename = __file__ + with self.assertRaises(IOError): + example.file_contents(filename) + self.fs.add_real_file(filename) + self.assertEqual(example.file_contents(filename), self.real_contents) if __name__ == "__main__": diff --git a/fake_filesystem_unittest_test.py b/fake_filesystem_unittest_test.py index 8518479d..59d1eada 100755 --- a/fake_filesystem_unittest_test.py +++ b/fake_filesystem_unittest_test.py @@ -191,22 +191,20 @@ def test_own_path_module(self): @unittest.skipIf(sys.version_info < (2, 7), "No byte strings in Python 2.6") class TestCopyOrAddRealFile(TestPyfakefsUnittestBase): - """Tests the `fake_filesystem_unittest.TestCase.copyRealFile()` method.""" + """Tests the `fake_filesystem_unittest.TestCase.copyRealFile()` method. + Note that `copyRealFile()` is deprecated in favor of `FakeFilesystem.add_real_file()`. + """ with open(__file__) as f: real_string_contents = f.read() with open(__file__, 'rb') as f: real_byte_contents = f.read() - # It is essential to do os.stat() after opening the real file, not before. - # This is because opening the file on MacOS and BSD updates access time - # st_atime. Windows offers the option to enable this behavior as well. real_stat = os.stat(__file__) def testCopyRealFile(self): - '''Copy a real file to the fake file system''' + '''Typical usage of deprecated copyRealFile()''' # Use this file as the file to be copied to the fake file system real_file_path = __file__ - fake_file_path = 'foo/bar/baz' - fake_file = self.copyRealFile(real_file_path, fake_file_path) + fake_file = self.copyRealFile(real_file_path) self.assertTrue('class TestCopyRealFile(TestPyfakefsUnittestBase)' in self.real_string_contents, 'Verify real file string contents') @@ -216,25 +214,29 @@ def testCopyRealFile(self): # note that real_string_contents may differ to fake_file.contents due to newline conversions in open() self.assertEqual(fake_file.byte_contents, self.real_byte_contents) - self.assertEqual(fake_file.st_mode, self.real_stat.st_mode) + self.assertEqual(oct(fake_file.st_mode), oct(self.real_stat.st_mode)) self.assertEqual(fake_file.st_size, self.real_stat.st_size) self.assertEqual(fake_file.st_ctime, self.real_stat.st_ctime) - self.assertEqual(fake_file.st_atime, self.real_stat.st_atime) + self.assertGreaterEqual(fake_file.st_atime, self.real_stat.st_atime) + self.assertLess(fake_file.st_atime, self.real_stat.st_atime + 10) self.assertEqual(fake_file.st_mtime, self.real_stat.st_mtime) self.assertEqual(fake_file.st_uid, self.real_stat.st_uid) self.assertEqual(fake_file.st_gid, self.real_stat.st_gid) - fake_file_path = '/nonexistent/directory/file' - with self.assertRaises(IOError): - self.copyRealFile(real_file_path, fake_file_path, - create_missing_dirs=False) - - def testCopyRealFileNoDestination(self): + def testCopyRealFileDeprecatedArguments(self): + '''Deprecated copyRealFile() arguments''' real_file_path = __file__ self.assertFalse(self.fs.Exists(real_file_path)) - self.copyRealFile(real_file_path) + # Specify redundant fake file path + self.copyRealFile(real_file_path, real_file_path) self.assertTrue(self.fs.Exists(real_file_path)) + # Test deprecated argument values + with self.assertRaises(ValueError): + self.copyRealFile(real_file_path, '/different/filename') + with self.assertRaises(ValueError): + self.copyRealFile(real_file_path, create_missing_dirs=False) + def testAddRealFile(self): '''Add a real file to the fake file system to be read on demand''' diff --git a/pyfakefs/fake_filesystem.py b/pyfakefs/fake_filesystem.py index 88bc4dc8..fab09e32 100644 --- a/pyfakefs/fake_filesystem.py +++ b/pyfakefs/fake_filesystem.py @@ -458,6 +458,8 @@ def byte_contents(self): self.contents_read = True with io.open(self.file_path, 'rb') as f: self._byte_contents = f.read() + # On MacOS and BSD, the above io.open() updates atime on the real file + self.st_atime = os.stat(self.file_path).st_atime return self._byte_contents def IsLargeFile(self): @@ -1772,14 +1774,15 @@ def CreateFile(self, file_path, st_mode=stat.S_IFREG | PERM_DEF_FILE, file_path, st_mode, contents, st_size, create_missing_dirs, apply_umask, encoding, errors) def add_real_file(self, file_path, read_only=True): - """Create file_path, including all the parent directories along the way, for a file - existing in the real file system without reading the contents, which will be read on demand. + """Create file_path, including all the parent directories along the way, for an existing + real file. The contents of the real file are read only on demand. New in pyfakefs 3.2. Args: - file_path: path to the existing file. - read_only: if set, the file is treated as read-only, e.g. a write access raises an exception; - otherwise, writing to the file changes the fake file only as usually. + file_path: Path to an existing file in the real file system + read_only: If `True` (the default), writing to the fake file + raises an exception. Otherwise, writing to the file changes + the fake file only. Returns: the newly created FakeFile object. @@ -1787,14 +1790,23 @@ def add_real_file(self, file_path, read_only=True): Raises: OSError: if the file does not exist in the real file system. IOError: if the file already exists in the fake file system. + + .. note:: On MacOS and BSD, accessing the fake file's contents will update \ + both the real and fake files' `atime.` (access time). In this \ + particular case, `add_real_file()` violates the rule that `pyfakefs` \ + must not modify the real file system. \ + \ + Further, Windows offers the option to enable atime, and older \ + versions of Linux may also modify atime. """ return self._CreateFile(file_path, read_from_real_fs=True, read_only=read_only) def add_real_directory(self, dir_path, read_only=True, lazy_read=True): - """Create fake directory for the existing directory at path, and entries for all contained - files in the real file system. + """Create a fake directory corresponding to the real directory at the specified + path. Add entries in the fake directory corresponding to the entries in the + real directory. New in pyfakefs 3.2. Args: @@ -1835,8 +1847,9 @@ def add_real_directory(self, dir_path, read_only=True, lazy_read=True): return new_dir def add_real_paths(self, path_list, read_only=True, lazy_dir_read=True): - """Convenience method to add several files and directories from the real file system - in the fake file system. See `add_real_file()` and `add_real_directory()`. + """This convenience method adds multiple files and/or directories from the + real file system to the fake file system. See `add_real_file()` and + `add_real_directory()`. New in pyfakefs 3.2. Args: @@ -1861,7 +1874,8 @@ def _CreateFile(self, file_path, st_mode=stat.S_IFREG | PERM_DEF_FILE, contents='', st_size=None, create_missing_dirs=True, apply_umask=False, encoding=None, errors=None, read_from_real_fs=False, read_only=True): - """Internal fake file creation, supports both normal fake files and fake files from real files. + """Internal fake file creator that supports both normal fake files and fake + files based on real files. Args: file_path: path to the file to create. diff --git a/pyfakefs/fake_filesystem_unittest.py b/pyfakefs/fake_filesystem_unittest.py index 1b08fa10..88b6126f 100644 --- a/pyfakefs/fake_filesystem_unittest.py +++ b/pyfakefs/fake_filesystem_unittest.py @@ -61,11 +61,6 @@ else: import builtins -REAL_OPEN = builtins.open -"""The real (not faked) `open` builtin.""" -REAL_OS = os -"""The real (not faked) `os` module.""" - def load_doctests(loader, tests, ignore, module, additional_skip_names=None, patch_path=True): # pylint: disable=unused-argument @@ -134,53 +129,39 @@ def patches(self): def copyRealFile(self, real_file_path, fake_file_path=None, create_missing_dirs=True): - """Copy the file `real_file_path` from the real file system to the fake - file system file `fake_file_path`. The permissions, gid, uid, ctime, - mtime and atime of the real file are copied to the fake file. - - This is a helper method you can use to set up your test more easily. + """Add the file `real_file_path` in the real file system to the same + path in the fake file system. - This method is available in Python 2.7 and above. + **This method is deprecated** in favor of :py:meth:`FakeFilesystem..add_real_file`. + `copyRealFile()` is retained with limited functionality for backward + compatability only. Args: - real_file_path: Path to the source file in the real file system. - fake_file_path: path to the destination file in the fake file system. - Defaults to real_file_path. - create_missing_dirs: if True, auto create missing directories. + real_file_path: Path to the file in both the real and fake file systems + fake_file_path: Deprecated. Use the default, which is `real_file_path`. + If a value other than `real_file_path` is specified, an `ValueError` + exception will be raised. + create_missing_dirs: Deprecated. Use the default, which creates missing + directories in the fake file system. If `False` is specified, an + `ValueError` exception is raised. Returns: - the newly created FakeFile object. + The newly created FakeFile object. Raises: - IOError: if the file already exists. - IOError: if the containing directory is required and missing. - - .. note:: The fake file's atime is the access time of the real file \ - before it accessed by `copyRealFile()`. Then, the real file \ - is opened in order to copy its contents, whereupon \ - **MacOS and BSD update the real file's atime** to the time \ - at which your test used `copyRealFile()`. \ - \ - Thus on these platforms atime is subject to Heisenberg's \ - uncertainty principle--by merely accessing the real file, \ - your test changes the real file's atime. Further, Windows \ - offers the option to enable atime, and older versions of \ - Linux may also modify atime. + IOError: If the file already exists in the fake file system. + ValueError: If deprecated argument values are specified + + See: + :py:meth:`FakeFileSystem.add_real_file` """ - if fake_file_path is None: - fake_file_path = real_file_path - real_stat = REAL_OS.stat(real_file_path) - with REAL_OPEN(real_file_path, 'rb') as real_file: - real_contents = real_file.read() - fake_file = self.fs.CreateFile(fake_file_path, st_mode=real_stat.st_mode, - contents=real_contents, - create_missing_dirs=create_missing_dirs) - fake_file.st_ctime = real_stat.st_ctime - fake_file.st_atime = real_stat.st_atime - fake_file.st_mtime = real_stat.st_mtime - fake_file.st_gid = real_stat.st_gid - fake_file.st_uid = real_stat.st_uid - return fake_file + if fake_file_path is not None and real_file_path != fake_file_path: + raise ValueError("CopyRealFile() is deprecated and no longer supports " + "different real and fake file paths") + if not create_missing_dirs: + raise ValueError("CopyRealFile() is deprecated and no longer supports " + "NOT creating missing directories") + return self._stubber.fs.add_real_file(real_file_path, read_only=False) def setUpPyfakefs(self): """Bind the file-related modules to the :py:class:`pyfakefs` fake file