Rclone fails uploads with some WebDAV servers #6145
Comments
Evengard
changed the title
|
Edit: WebDAV is the old API. I don't think that we want to support it. If @ncw agrees then this issue should be closed as “won't do” in favor of #6032. Just in case we want to revisit this in the future, I left my Yandex WebDAV exploration notes below: Relevant code: where res.getParams().setParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, true);
res.getParams().setParameter(CoreProtocolPNames.WAIT_FOR_CONTINUE, timeout);
In our case: ( It does a see implem of If Unrelated but interesting, here is how the different HTTP error codes are handled: https://github.com/yandex-disk/yandex-disk-sdk-java/blob/bc2e0e50ef7a2ffa11af2439599d0b136474581b/yandex-disk-sdk/src/com/yandex/disk/client/TransportClient.java#L322-L355 |
The associated forum post URL from
https://forum.rclone.orgNot applicable
What is the problem you are having with rclone?
While this issue may seem related to #6032, I'd classify it as a separate issue, as such a case may be happening with any other WebDAV server.
Yandex.Disk does provide a WebDAV endpoint, and I was testing it using rclone. While uploading any file with any significant size (for me it was about 50 MiB or anything more), it seems the transfer is getting stuck after transferring the first 40-50 MiB or around that, then doing multiple internal retries and subsequently failing the transfer alltogether.
My assumption is that (according to https://yandex.ru/dev/disk/doc/dg/reference/put.html) the server is expected to reply with 100 Continue to the PUT request, and sending data before that seems to actually break the transfer (although can't really confirm it, as no answer is actually displayed in
--dump headers)What is your rclone version (output from
rclone version)Which OS you are using and how many bits (e.g. Windows 7, 64 bit)
ArchLinux amd64
Which cloud storage system are you using? (e.g. Google Drive)
WebDAV endpoint of Yandex.Disk, configured as WebDAV.
The command you were trying to run (e.g.
rclone copy /tmp remote:tmp)rclone copy -vv --dump headers /data/NWRESTOREFORUM.7z ywd:/2tbThe NWRESTOREFORUM.7z is a file of about 400 MiB, never finishing this transfer successfully.
A log from the command with the
-vvflag (e.g. output fromrclone -vv copy /tmp remote:tmp)Please note the latest INFO entry on the log: it is the moment where the whole transfer is actually restarted instead of uploading another chunk. It just was lucky to upload a bit more. On other tries it may had uploaded even less. I think it may be connected to the actual internet speed - "how much data the uplink is able to send before receiving this response 100 Continue".
How to use GitHub
The text was updated successfully, but these errors were encountered: