Graceful handling of Timeout

[tvandera]

Bug Report Details

Describe the bug
When running onedrive --synchronize, a timeout sometimes happens. It would be nice
if onedrive could retry.

Application and Operating System Details:

• OS: Linux zcu102-host 4.4.0-104-generic Gateway Timeout - JSONValue is not an object #127~14.04.1-Ubuntu SMP Mon Dec 11 12:44:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• Application version: onedrive v1.1.1-112-ga491620

Complete Verbose Log Output

Uploading file ./xxxxx(censored) ...
 done.
std.net.curl.CurlTimeoutException@std/net/curl.d(4329): Timeout was reached on handle 206B5B0
----------------
??:? pure @safe void std.exception.bailOut!(std.net.curl.CurlTimeoutException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x625102]
??:? pure @safe bool std.exception.enforce!(std.net.curl.CurlTimeoutException).enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x625075]
??:? void std.net.curl.Curl._check(int) [0x5f77c5]
??:? int std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) [0x5f7c24]
??:? int std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) [0x5f66e0]
src/onedrive.d:426 std.json.JSONValue onedrive.OneDriveApi.perform() [0x5a177f]
src/onedrive.d:311 std.json.JSONValue onedrive.OneDriveApi.get(const(char)[], bool) [0x5a1036]
src/onedrive.d:188 std.json.JSONValue onedrive.OneDriveApi.getPathDetails(const(immutable(char)[])) [0x5a053f]
src/sync.d:1105 void sync.SyncEngine.uploadNewFile(immutable(char)[]) [0x5aefff]
src/sync.d:985 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x5ae734]
src/sync.d:976 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x5ae62b]
src/sync.d:976 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x5ae62b]
src/sync.d:976 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x5ae62b]
src/sync.d:976 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x5ae62b]
src/sync.d:976 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x5ae62b]
src/sync.d:723 void sync.SyncEngine.scanForDifferences(immutable(char)[]) [0x5ad411]
src/main.d:388 void main.performSync(sync.SyncEngine, immutable(char)[], bool, bool, bool) [0x5923e3]
src/main.d:254 _Dmain [0x591d0e]

Additional context
After restarting onedrive, it works fine again, until the next timeout.

[abraunegg]

Hi,

Your currently using the skilion codebase: onedrive v1.1.1-112-ga491620 which contains significant defects including this one.

Please retest using my branch & if the same issue occurs please open a new case.

[tvandera]

@abraunegg: Hmm, that's weird, I pulled from your master.

$ git log -1
commit a491620c1a188bfc40d454f689bbf5b1b2539e68
Author: abraunegg <alex.braunegg@gmail.com>
Date:   Thu Jul 19 06:43:30 2018 +1000

    Update logging message for single directory removal

    * Update logging message for single directory removal - we are trying to delete not create

$ git describe --tags
v1.1.1-112-ga491620

[tvandera]

Hey Alex,

I'm pretty confident I'm using your version, more specifically this commit:
a491620

Cheers,
Tom

[abraunegg]

Hi Tom,

There must be something screwed up somewhere for you as I just did the following:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
make
sudo make install

Which gave me the following output:

Cloning into 'onedrive'...
remote: Counting objects: 1395, done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 1395 (delta 34), reused 51 (delta 25), pack-reused 1325
Receiving objects: 100% (1395/1395), 444.09 KiB | 259.00 KiB/s, done.
Resolving deltas: 100% (942/942), done.

echo v2.0.2-3-ge58c76d >version
dmd -g -ofonedrive -O -L-lcurl -L-lsqlite3 -L-ldl -J. src/config.d src/itemdb.d src/log.d src/main.d src/monitor.d src/onedrive.d src/qxor.d src/selective.d src/sqlite.d src/sync.d src/upload.d src/util.d
sed "s|@PREFIX@|/usr/local|g" systemd.units/onedrive.service.in > onedrive.service
sed "s|@PREFIX@|/usr/local|g" systemd.units/onedrive@.service.in > onedrive@.service

git describe --tags
v2.0.2-3-ge58c76d

[tvandera]

Hello Alex,
Here's how the git describe is still showing v1.1.1

I started from @skilion's master branch

$ git clone git@github.com:skilion/onedrive.git
Cloning into 'onedrive'...
remote: Counting objects: 961, done.
remote: Total 961 (delta 0), reused 0 (delta 0), pack-reused 961
Receiving objects: 100% (961/961), 283.33 KiB | 932.00 KiB/s, done.
Resolving deltas: 100% (667/667), done.
$ cd onedrive/
$ git describe --tags
v1.1.1-2-gc231b13

and merged your master into that. This did not pull in your tags:

$ git remote add abraunegg git@github.com:abraunegg/onedrive.git
$ git pull abraunegg master
remote: Counting objects: 425, done.
remote: Total 425 (delta 79), reused 79 (delta 79), pack-reused 346
Receiving objects: 100% (425/425), 141.90 KiB | 518.00 KiB/s, done.
Resolving deltas: 100% (275/275), completed with 16 local objects.
From github.com:abraunegg/onedrive
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> abraunegg/master
Updating c231b13..e58c76d
Fast-forward
 .github/ISSUE_TEMPLATE/bug_report.md                     |  36 +++++
 <part of output removed>
$ git describe --tags
v1.1.1-114-ge58c76d

This is OK since there are no commits that are in skilion/master but not in your master:

$ git log origin/master ^abraunegg/master

[abraunegg]

No worries - that then explains the version issue.

Skilion's branch is pretty much dead - no response on items, a few minor updates early this year - but no bug fixes or response in > 12 months, so worth just using my fork as the master.

[abraunegg]

Can you help provide some environmental details around your issue?

• Internet bandwidth (upstream / downstream)
• Size of file attempting to upload
• Personal or Business account
• Ping time response to Microsoft OneDrive services - graph.microsoft.com
• Any sort of firewall / web content security device between you and Microsoft OneDrive services
• Please check your Internet access for dropouts of connectivity at the same date & timestamp when the Timeout was reached on handle error occurred.

This will help me 'simulate' your environment locally & attempt to replicate the issue in order to work on a fix.

[abraunegg]

@tvandera
Can you help to test PR #86?

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/86/head:pr86
git checkout pr86
make
make install

[tvandera]

Here are the details you requested:

• internet speed: 93 Mbit/s Up, 93 Mbit/s down (computer happens to be on a 100Mbit/s switch)
• size of upload: 300GB
• Bussiness (Office365) account
• Ping time to graph.microsoft.com: does not ping back (timeout)
• No special firewall/filter between me and the internet AFAIK

[tvandera]

Pulled your pr86 branch, onedrive v2.0.2-5-ga58ceeb is happily churning away.
I will let you know if I get Curl timeouts.

[abraunegg]

Thanks - I suspect some sort of ISP disruption occurred which is why the timeout occured - if you have any logs to help validate that theory that would be great.

[abraunegg]

@tvandera
Any update as to your testing of PR #86 ?

[tvandera]

Sync is running fine. No timeouts so far. Nothing in onedrive.log about Timeouts.

[abraunegg]

Thanks for the update. Will keep this issue open for another week - if all OK after that, will merge the PR & close the issue.

[abraunegg]

PR #86 merged. Closing issue.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.