Skip to content

Conversation

@ianthomas23
Copy link
Collaborator

Fixes #1232.

This is for a non-recursive copy of an empty directory such as

cp("source/somedir/", "target/", recursive=False)

which is now a no-op to agree with command-line copy.

There were two possible approaches. The one I've opted for is to leave expand_path() as it is, returning the empty directory, and checking this separately in cp, get and put to identify the "do nothing" scenario and return early.

The other approach would have been to add a new kwarg to expand_path() to override the current withdirs=True. This would allow an empty return from the function which would have needed extra handling so I considered this a worse solution than the one I have adopted. I am writing this here because I suspect in the long run this solution may be required to support corner cases of cp passing a list of individual files and empty directories, which probably nobody is doing but must eventually be supported. Very low priority though.

# always be a forward slash, simplifying this function.
# See https://github.com/fsspec/filesystem_spec/pull/1250
return path.endswith(os.sep) or os.altsep and path.endswith(os.altsep)
return path.endswith(os.sep) or (os.altsep is not None and path.endswith(os.altsep))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wasn't quite correct before but now is.

]

fs.rm(target + "/empty", recursive=True)
assert fs.find(target, withdirs=True) == []
Copy link
Collaborator Author

@ianthomas23 ianthomas23 May 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is effectively duplicated in the abstract test harness, so it can eventually be removed. But I have fixed it here for the time being.

The same applies to the one in test_memory.py below as well.

# Non-recursive copy of directory does nothing.
return
source_is_str = isinstance(rpath, str)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reordered the code here (and the other 2 functions below) to separate out the handling of source and target paths, with a line of whitespace between, for increased clarity.

@ianthomas23
Copy link
Collaborator Author

s3fs-pytest test failure is the same one we are seeing in the s3fs repo itself (e.g. https://github.com/fsspec/s3fs/actions/runs/4852093306/jobs/8650821223?pr=731).

@martindurant
Copy link
Member

test failure is the same one we are seeing in the s3fs repo

OK.

@martindurant
Copy link
Member

Thorough code, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Non-recursive directory copy incorrect

2 participants