put() inconsistent with cp

[kamalsharma2]

The behavior of put is not consistent with the linux command cp. When we have source and target paths like
source : "path1/file1.txt"
target : "path2/"
and Target directory already exists, then with linux command cp -r path1/* path2, we get
target : "path2/file1.txt"
However, doing the same with fs.put("path1/*","path2",recursive=True), we get
target : "path2/path1/file1.txt"

To remove this inconsistency, '/*' case should be handled in put()

[ianthomas23]

Returning to this now that we have cp, get and put behaviour consistent with command-line cp (PR #1148). The desired (correct) behaviour for the example given can be obtained using

fs.put("path1/", "path2", recursive=True)

i.e. by removing the wildcard *.

With the wildcard present the behaviour isn't correct, it is the same as fs.put("path1", ...) rather than fs.put("path1/", ...). Possible fix for this is to change in about 6 calls to other_paths the exists argument which controls whether files should end up within the target directory or not. Currently this is controlled by

not path.endswith("/")

which we could change to

not path.endswith(("/", "/*"))

I will try this out and see if it breaks anything else.

[ianthomas23]

My previous suggestion isn't sufficient as there are other parts of the code which handle a trailing slash differently, so removing the asterisk needs to occur earlier. So now I think that

fs.put("path1/*", "path2")

should be handled exactly the same as

fs.put("path/", "path2", recursive=True, maxdepth=1)

This means that when we identify and drop the trailing slash, we need to force the recursive and maxdepth values. We have to be careful if the user specifies a specific value for maxdepth such as

fs.put("path1/*", "path2", maxdepth=3)

as the asterisk and the maxdepth are not really consistent here. We could disallow such combinations, but I think the best behaviour here is to only override the maxdepth if the user hasn't already specified it. If they have specified it then we have to respect their explicit instruction.

Bringing this together the new logic (for cp, get and put) looks something like

# Existing code: if lpath is a string then convert it to posix form
if lpath.endswith("/*"):
    lpath = lpath[:-1]
    recursive = True
    if not maxdepth:
        maxdepth = 1

[ianthomas23]

This is now fixed by #1267 and previous PRs mentioned therein. cp, get and put are now consistent.

Documentation for the expected behaviour of cp/get/put is at https://filesystem-spec.readthedocs.io/en/latest/copying.html.