Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs.copy.copy_*_if #460

Merged
merged 10 commits into from Apr 8, 2021
Merged

fs.copy.copy_*_if #460

merged 10 commits into from Apr 8, 2021

Conversation

atollk
Copy link
Member

@atollk atollk commented Mar 20, 2021

Type of changes

  • New feature
  • Tests

Checklist

  • I've run the latest black with default args on new code.
  • I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate.
  • I've added tests for new code.
  • I accept that @PyFilesystem/maintainers may be pedantic in the code review.

Description

Fixes #458

Added _copy_if functions to fs.copy module.

The free functions copy_file, copy_file_if_newer, copy_dir, copy_dir_if_newer, copy_fs, and copy_fs_if_newer now are all implemented in terms of copy_file_if, copy_dir_if, and copy_fs_if.

This change streamlined the code by removing duplication in logic and clustering similar logic closer together.

Also, though this has not been tested, copy_dir_if_newer should be faster in this implementation when copying to a dst_dir that contains a lot of files that are not present in the respective src_dir.

Finally, a bug was fixed that was caused by a false lookup in the dictionary of file infos from dst_fs. This fix could cause unnecessary copies and therefore a decrease in performance.

atollk added 3 commits March 19, 2021 22:21
The free functions copy_file, copy_file_if_newer, copy_dir, copy_dir_if_newer, copy_fs, and copy_fs_if_newer now are all implemented in terms of copy_file_if, copy_dir_if, and copy_fs_if.
Unit Tests for this change will be created in a future commit.

This change streamlined the code by removing duplication in logic and clustering similar logic closer together.

Also, though this has not been tested, copy_dir_if_newer should be faster in this implementation when copying to a dst_dir that contains a lot of files that are not present in the respective src_dir.

Finally, a bug was fixed that was caused by a false lookup in the dictionary of file infos from dst_fs. This fix could cause unnecessary copies and therefore a decrease in performance.
…s.copy.

tests/test_copy experienced a general rework in many areas. Before, tests focused on the most basic of test cases and thus missed more complex situations that could (and actually did) cause errors. Thus, some tests for the same unit were merged to create more relevant test cases.
@atollk
Copy link
Member Author

atollk commented Mar 20, 2021

Imho, copy_*_if_newer should be marked as deprecated if this change is merged.

@coveralls
Copy link

coveralls commented Mar 20, 2021

Coverage Status

Coverage decreased (-0.08%) to 95.277% when pulling 2d04cca on atollk:issue_458 into f6a6195 on PyFilesystem:master.

@althonos
Copy link
Member

Nice work! I'll wait for your unit tests to be added.

I'm thinking we should soft-deprecate the copy_if_newer functions eventually, since they are just aliasing now, and it's confusing for the end user that we provide them for some conditions (newer) but not for the remaining ones.

fs/copy.py Outdated Show resolved Hide resolved
fs/copy.py Outdated Show resolved Hide resolved
fs/copy.py Show resolved Hide resolved
@atollk
Copy link
Member Author

atollk commented Mar 22, 2021

@althonos Thanks. I already added unit tests, I just forgot to remove that part from the description, sorry. It looks like the lines that are not covered anymore are the _if_newer functions which are just one-line wrappers now.

Copy link
Member

@althonos althonos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small fixes related to the deprecation of older functions, and we should be good to go!

fs/copy.py Outdated Show resolved Hide resolved
fs/copy.py Outdated Show resolved Hide resolved
fs/copy.py Outdated Show resolved Hide resolved
fs/copy.py Outdated Show resolved Hide resolved
@althonos althonos added this to the v2.5.0 milestone Apr 8, 2021
@althonos althonos merged commit 5f73778 into PyFilesystem:master Apr 8, 2021
@althonos
Copy link
Member

althonos commented Apr 8, 2021

Thank you! First PR out of the way 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add more options than "if_newer" to fs.copy
3 participants