Async cp/get/put the same as sync behaviour#1163
Conversation
|
The CIs pull from the other repos main/master branches, but could be set temporarily to the incoming branch. Im not actually worried if CI shows red for a day or more while the different components land, so maybe best not to change the CI settings. Having made all these changes, it would be interesting to compare with what generic.rsync does for similar situations. |
|
Also, the green ✅ suggests that gcsfs needs more tests, we should have expected something to break! |
I'll take a look, and also check what command-line |
|
:) our rsync doesn't really get close to the many options of gnu rsync |
|
The code here looks good, I am happy to merge when you say you are done, and the changes in gcsfs and s3fs can follow. @hayesgb @TomAugspurger - this may affect adlfs |
I am done here, and happy for this to be merged. |
This PR implements the correct
_cp,_getand_putbehaviour forAsyncFileSystemthat PR #1148 implemented for the synchronousAbstractFileSystem. The changes are all inasyn.pyand are almost identical to those made inspec.pyexcept where a check for a trailing slash needs to be made earlier on as a subsequent call to_strip_protocolalso strips the trailing slash.There are no tests added here, instead I will shortly submit PRs to both
s3fsandgcsfsto add them in line with the similar tests made in PR #1148. The tests I have added to thoses3fsandgcsfsbranches all fail when run against the latest commit offsspecand pass when run against this PR. We will need to coordinate the CI/merging of the three PRs before we can be completely sure that everything consistently works together.