Dropbox stream download with RetryWrapper

[PVince81]

• No more temporary file on download
• Less likeliness of timeouts for big files
• More snappy download experience

To test:

1. Setup Dropbox ext storage as "/dropbox"
2. Setup desktop client
3. Copy a big file (ex: 65MB) in "/dropbox/test.mp4"
4. Wait for sync
5. Close sync client
6. Reset sync client
7. Start sync client
8. Wait for sync download
9. Compare md5sum with original file

• TEST: download big file without encryption
• TEST: download big file with encryption
• TEST: range request still works curl -X GET -H Range:bytes=2-5 'http://admin:admin@localhost/owncloud/remote.php/webdav/dropbox/bacon.txt'

@icewind1991 @SergioBertolinSG @Xenopathic

[mention-bot]

By analyzing the blame information on this pull request, we identified @MTGap, @icewind1991, @butonic and @Xenopathic to be potential reviewers

[PVince81]

Supersedes #23372 where it was observed that the temp file is increasing likeliness of timeouts.

@karlitschek backport to stable9 ?

[PVince81]

• TEST: no unexpected unit test failed (compared to the known ones from Failing dropbox unit tests #19868)

[PVince81]

Fixes #4040

[icewind1991]

👍 looks good

[karlitschek]

please backport 👍

[PVince81]

stable9: #23522

[PVince81]

@davitol @SergioBertolinSG can you guys test downloads of small and big files for this PR and also the backport #23522 for 9.0.1 ? Thanks

[davitol]

1. Close sync client
2. Reset sync client
3. Start sync client

@PVince81 Did you mean close and restart sync client? Or maybe you meant to delete the account and the local files and configure again the server from the beginning?

Tested closing and restarting desktop client, no problems were found.

[davitol]

👎 weird behaviour uploading and downloading files They appear/disappear without any reason

{"reqId":"+GTax6T+vuwydCV3Mt4V","remoteAddr":"82.159.139.58","app":"files_external","message":"Resource at uri: https:\/\/api.dropbox.com\/1\/fileops\/delete could not be found","level":3,"time":"2016-03-29T15:37:06+00:00","method":"DELETE","url":"\/remote.php\/webdav\/Dropbox\/video.mp4","user":"admin"}
{"reqId":"j89ag7QM2CxkTcKGzO8F","remoteAddr":"82.159.139.58","app":"PHP","message":"fseek(): stream does not support seeking at \/opt\/owncloud\/apps\/files_external\/3rdparty\/icewind\/streams\/src\/Wrapper.php#74","level":3,"time":"2016-03-29T15:52:09+00:00","method":"GET","url":"\/remote.php\/webdav\/Dropbox\/video.mp4","user":"admin"}
{"reqId":"bp2Kwo46MIuetyf3zJeI","remoteAddr":"82.159.139.58","app":"PHP","message":"fseek(): stream does not support seeking at \/opt\/owncloud\/apps\/files_external\/3rdparty\/icewind\/streams\/src\/Wrapper.php#74","level":3,"time":"2016-03-29T15:54:35+00:00","method":"GET","url":"\/remote.php\/webdav\/Dropbox\/video.mp4","user":"admin"}

{"reqId":"McogYKQoqXEls56cmGP5","remoteAddr":"82.159.139.58","app":"files_external","message":"Resource at uri: https:\/\/api.dropbox.com\/1\/fileops\/delete could not be found","level":3,"time":"2016-03-29T16:24:07+00:00","method":"DELETE","url":"\/remote.php\/webdav\/Dropbox\/video.mp4","user":"admin"}
{"reqId":"vUqTrxnQaLwSCWn9ZQn+","remoteAddr":"82.159.139.58","app":"files_external","message":"Resource at uri: https:\/\/api.dropbox.com\/1\/fileops\/delete could not be found","level":3,"time":"2016-03-29T16:28:28+00:00","method":"MOVE","url":"\/remote.php\/webdav\/Dropbox\/NEWS-992.MOV","user":"admin"}
{"reqId":"vUqTrxnQaLwSCWn9ZQn+","remoteAddr":"82.159.139.58","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 423 \\\"Dropbox\\\/NEWS-992.MOV\\\" is locked\",\"Exception\":\"OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Exception\\\\FileLocked\",\"Code\":0,\"Trace\":\"#0 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(640): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->move('Dropbox\\\/NEWS-99...', 'Dropbox\\\/NEWS-99...')\\n#1 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpMove(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#2 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#3 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:MOVE', Array)\\n#4 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#5 \\\/opt\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(55): Sabre\\\\DAV\\\\Server->exec()\\n#6 \\\/opt\\\/owncloud\\\/remote.php(138): require_once('\\\/opt\\\/owncloud\\\/a...')\\n#7 {main}\",\"File\":\"\\\/opt\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/connector\\\/sabre\\\/objecttree.php\",\"Line\":248,\"User\":\"admin\"}","level":4,"time":"2016-03-29T16:28:33+00:00","method":"MOVE","url":"\/remote.php\/webdav\/Dropbox\/NEWS-992.MOV","user":"admin"}
{"reqId":"MvMaAj5KoV+wZ6pJjIKI","remoteAddr":"82.159.139.58","app":"PHP","message":"fseek(): stream does not support seeking at \/opt\/owncloud\/apps\/files_external\/3rdparty\/icewind\/streams\/src\/Wrapper.php#74","level":3,"time":"2016-03-29T16:31:57+00:00","method":"GET","url":"\/remote.php\/webdav\/Dropbox\/NEWS-992.mov","user":"admin"}
{"reqId":"yM+lKnALwYyZPOsrK6LH","remoteAddr":"82.159.139.58","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 416 The start offset (137986728) exceeded the size of the entity (94675284)\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\RequestedRangeNotSatisfiable\",\"Code\":0,\"Trace\":\"#0 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpGet(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#1 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#2 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:GET', Array)\\n#3 \\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#4 \\\/opt\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(55): Sabre\\\\DAV\\\\Server->exec()\\n#5 \\\/opt\\\/owncloud\\\/remote.php(138): require_once('\\\/opt\\\/owncloud\\\/a...')\\n#6 {main}\",\"File\":\"\\\/opt\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php\",\"Line\":158,\"User\":\"admin\"}","level":4,"time":"2016-03-29T16:33:30+00:00","method":"GET","url":"\/remote.php\/webdav\/Dropbox\/NEWS-992.mov","user":"admin"}
{"reqId":"O5c+\/1vksndWM+NSTaGS","remoteAddr":"82.159.139.58","app":"PHP","message":"fseek(): stream does not support seeking at \/opt\/owncloud\/apps\/files_external\/3rdparty\/icewind\/streams\/src\/Wrapper.php#74","level":3,"time":"2016-03-29T16:41:27+00:00","method":"GET","url":"\/remote.php\/webdav\/Dropbox\/Firefox%2040.0.3.dmg","user":"admin"}
{"reqId":"c4AtP3ShaT4AW9AJugK7","remoteAddr":"82.159.139.58","app":"files_external","message":"Resource at uri: https:\/\/api.dropbox.com\/1\/fileops\/delete could not be found","level":3,"time":"2016-03-29T16:43:36+00:00","method":"DELETE","url":"\/remote.php\/webdav\/Dropbox\/Firefox%2040.0.3.dmg","user":"admin"}

[PVince81]

fseek ? Was it when testing encryption ?

[davitol]

No, it wasn't encrypted yet, but your comment rang me a bell related to if there are some files in the dropbox account that were encrypted before with other server, there could be problems. Let me try again with an empty Dropbox account. I'll write down here the test results.

[davitol]

Let me try again with an empty Dropbox account.

With an empty Dropbox account it seems to work fine 👍

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.