Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-73991: Support preserving metadata in pathlib.Path.copy() #120806

Merged
merged 15 commits into from
Jul 6, 2024

Conversation

barneygale
Copy link
Contributor

@barneygale barneygale commented Jun 20, 2024

Add preserve_metadata keyword-only argument to pathlib.Path.copy(), defaulting to false. When set to true, we copy timestamps, permissions, extended attributes and flags where available, like shutil.copystat(). The argument has no effect on Windows, where metadata is always copied.

Internally (in the pathlib ABCs), path types gain _readable_metadata and _writable_metadata attributes. These sets of strings describe what kinds of metadata can be retrieved and stored. We take an intersection of source._readable_metadata and target._writable_metadata to minimise reads/writes. A new _read_metadata() method accepts a set of metadata keys and returns a dict with those keys, and a new _write_metadata() method accepts a dict of metadata. We might make these public in future, but it's hard to justify while the ABCs are still private.


📚 Documentation preview 📚: https://cpython-previews--120806.org.readthedocs.build/

Add *preserve_metadata* keyword-only argument to `pathlib.Path.copy()`,
defaulting to false. When set to true, we copy timestamps, permissions,
extended attributes and flags where available, like `shutil.copystat()`.
The argument has no effect on Windows, where metadata is always copied.

In the pathlib ABCs we copy the file permissions with `PathBase.chmod()`
where supported. In the future we might want to support a more generic
public interface for copying metadata between different types of `PathBase`
object, but it would be premature here.
@barneygale barneygale merged commit 88fc065 into python:main Jul 6, 2024
33 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang Installed 3.x has failed when building commit 88fc065.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/451/builds/4646) and take a look at the build logs.
  4. Check if the failure is related to this commit (88fc065) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/451/builds/4646

Failed tests:

  • test_pathlib

Failed subtests:

  • test_copy_file_preserve_metadata - test.test_pathlib.test_pathlib.PathSubclassTest.test_copy_file_preserve_metadata
  • test_copy_file_preserve_metadata - test.test_pathlib.test_pathlib.PathTest.test_copy_file_preserve_metadata
  • test_copy_file_preserve_metadata - test.test_pathlib.test_pathlib.PosixPathTest.test_copy_file_preserve_metadata

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.14/test/test_pathlib/test_pathlib.py", line 660, in test_copy_file_preserve_metadata
    os.setxattr(source, b'user.foo', b'42')
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 95] Operation not supported: '/tmp/test_python_pfjupu1y/test_python_2060648æ/@test_2060648_tmpæ/fileA'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.14/test/test_pathlib/test_pathlib.py", line 660, in test_copy_file_preserve_metadata
    os.setxattr(source, b'user.foo', b'42')
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 95] Operation not supported: '/tmp/test_python__riczf75/test_python_2046711æ/@test_2046711_tmpæ/fileA'

barneygale added a commit to barneygale/cpython that referenced this pull request Jul 6, 2024
Follow-up to python#120806. Use `os_helper.skip_unless_xattr` to skip testing
xattr preservation when unsupported.
barneygale added a commit that referenced this pull request Jul 7, 2024
Follow-up to #120806. Use `os_helper.skip_unless_xattr` to skip testing
xattr preservation when unsupported.
noahbkim pushed a commit to hudson-trading/cpython that referenced this pull request Jul 11, 2024
…ython#120806)

Add *preserve_metadata* keyword-only argument to `pathlib.Path.copy()`, defaulting to false. When set to true, we copy timestamps, permissions, extended attributes and flags where available, like `shutil.copystat()`. The argument has no effect on Windows, where metadata is always copied.

Internally (in the pathlib ABCs), path types gain `_readable_metadata` and `_writable_metadata` attributes. These sets of strings describe what kinds of metadata can be retrieved and stored. We take an intersection of `source._readable_metadata` and `target._writable_metadata` to minimise reads/writes. A new `_read_metadata()` method accepts a set of metadata keys and returns a dict with those keys, and a new `_write_metadata()` method accepts a dict of metadata. We *might* make these public in future, but it's hard to justify while the ABCs are still private.
noahbkim pushed a commit to hudson-trading/cpython that referenced this pull request Jul 11, 2024
…thon#121444)

Follow-up to python#120806. Use `os_helper.skip_unless_xattr` to skip testing
xattr preservation when unsupported.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants