std.json.JSONException@std/json.d(508): Key not found: expirationDateTime #174

fgauvrit · 2018-09-14T06:56:00Z

Bug Report Details

Since the last onedrive client update I have the following error at the start : "std.json.JSONException@std/json.d(508): Key not found: expirationDateTime" and it makes a loop.

Application and Operating System Details:

OS: Linux FGA-E6540.HOME 4.4.143-65-default Fix config folder. #1 SMP Tue Aug 14 09:18:29 UTC 2018 (4e090cc) x86_64 x86_64 x86_64 GNU/Linux (OpenSUSE Leap 42.3)
Are you using a headless system (no gui) or with a gui installed? gui (Gnome 3.20.2)
Application version: onedrive v2.1.2-5-g53cbc8f
OneDrive Account Type : Onedrive Premium
DMD or LDC compiler version : DMD64 D Compiler v2.082.0

To Reproduce
With Systemd implementation, I am using the following service file :

[Unit]
Description=OneDrive Free Client
Documentation=https://github.com/abraunegg/onedrive
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/local/bin/onedrive --monitor --local-first --upload-only --skip-symlinks --verbose
Restart=on-failure
RestartSec=3

[Install]
WantedBy=default.target

Complete Verbose Log Output
With onedrive --synchronize --local-first --upload-only --skip-symlinks --verbose, I get the same issue. Please find here the attached log :
fgauvrit.onedrive.log.zip

Application Log Output:

sept. 14 08:34:28 FGA-E6540.HOME systemd[2634]: Started OneDrive Free Client.
sept. 14 08:34:28 FGA-E6540.HOME onedrive[23474]: Loading config ...
sept. 14 08:34:28 FGA-E6540.HOME onedrive[23474]: Using Config Dir: /home/fgauvrit/.config/onedrive
sept. 14 08:34:28 FGA-E6540.HOME onedrive[23474]: Initializing the OneDrive API ...
sept. 14 08:34:28 FGA-E6540.HOME onedrive[23474]: Opening the item database ...
sept. 14 08:34:28 FGA-E6540.HOME onedrive[23474]: All operations will be performed in: /home/data/Medias
sept. 14 08:34:28 FGA-E6540.HOME onedrive[23474]: Initializing the Synchronization Engine ...
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: Account Type: personal
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: Default Drive ID: 51e9aeceea51cd05
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: Default Root ID: 51E9AECEEA51CD05!117
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: Remaining Free Space: 718773188494
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: Fetching details for OneDrive Root
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: OneDrive Root exists in the database
sept. 14 08:34:31 FGA-E6540.HOME onedrive[23474]: Trying to restore the upload session ...
sept. 14 08:34:32 FGA-E6540.HOME onedrive[23474]: Continuing the upload session ...
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: [229B blob data]
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: std.json.JSONException@std/json.d(508): Key not found: expirationDateTime
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: ----------------
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: /usr/include/dlang/dmd/std/exception.d:515 pure @safe void std.exception.bailOut!(std.json.JSONException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x59fdbb]
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: ??:? pure @safe inout(std.json.JSONValue)* std.exception.enforce!(std.json.JSONException).enforce!(inout(std.json.JSONValue)*).enforce(inout(std.json.JSONValue)*, lazy const(char)[], immutable(char)[], ulong) [0x6484e2]
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: ??:? inout pure ref @safe inout(std.json.JSONValue) std.json.JSONValue.opIndex(immutable(char)[]) [0x5fa945]
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: src/upload.d:121 std.json.JSONValue upload.UploadSession.upload() [0x5cf77b]
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: src/sync.d:189 void sync.SyncEngine.init() [0x5c88d4]
sept. 14 08:34:42 FGA-E6540.HOME onedrive[23474]: src/main.d:242 _Dmain [0x5ad2b1]
sept. 14 08:34:42 FGA-E6540.HOME systemd[2634]: onedrive.service: Main process exited, code=exited, status=1/FAILURE
sept. 14 08:34:42 FGA-E6540.HOME systemd[2634]: onedrive.service: Unit entered failed state.
sept. 14 08:34:42 FGA-E6540.HOME systemd[2634]: onedrive.service: Failed with result 'exit-code'.
sept. 14 08:34:45 FGA-E6540.HOME systemd[2634]: onedrive.service: Service RestartSec=3s expired, scheduling restart.
sept. 14 08:34:45 FGA-E6540.HOME systemd[2634]: Stopped OneDrive Free Client.
sept. 14 08:34:45 FGA-E6540.HOME systemd[2634]: Started OneDrive Free Client.

The text was updated successfully, but these errors were encountered:

abraunegg · 2018-09-14T22:05:12Z

It appears that this has to do with the response on re-connecting to a session to re-upload.

Will investigate a resolve today

abraunegg · 2018-09-14T22:35:17Z

@fgauvrit
Whilst I cannot replicate this exact issue, as this is around OneDrive generating an error on a session resume, I am hoping you still have the issue & can test this PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/176/head:pr176
git checkout pr176
make
make install

If you can let me know that would be greatly appreciated.

abraunegg · 2018-09-20T19:47:05Z

@fgauvrit
Can you please provide an update here?

pglez82 · 2018-10-03T11:00:13Z

I am getting the same exception and I have tested the PR with no luck. I still get the same exception. Can I help you debug the problem?

pglez82 · 2018-10-03T11:25:46Z

I have checking the code and I think the problem is in the file .config/resume_upload. In this file I have:

{"error":{"code":"UnknownError","innerError":{"date":"2018-10-03T07:38:21","request-id":"1e95dd5f-9be3-4b64-9d64-b0811ad18ab2"},"message":""},"localPath":".\/Mis imágenes\/Viajes\/Mallorca 2014\/DSCN1244.JPG"}

so then the error happens when trying to restore the session and look for the 'expirationDateTime' attribute in the Json. Obviously there is no such parameter there. Hope this helps to track the bug.

abraunegg · 2018-10-03T20:27:00Z

@pglez82
Do you have some exact reproduction steps to tigger this issue?

PR #176 should try the resume, and then fail gracefully. If I can be given an exact reproducer I can diagnose further as to why the current patch does not work

pglez82 · 2018-10-03T20:42:01Z

@abraunegg
Sorry but I cant give reproduction steps as it happened when uploading all my files from my computer to onedrive for the first time (~100Gb of data). It happened after stopping (ctrl+c) the process and resuming it a few times.

abraunegg · 2018-10-03T20:44:03Z

@pglez82
Ok .. no problem .. will try and work something out locally to try and get a reproducer

pglez82 · 2018-10-03T20:45:05Z

as a workaround solution I have renamed the file .config/onedrive/resume_upload to .config/onedrive/resume_upload.bak and then I am able to continue the upload just fine

* Update fix for issue #174

abraunegg · 2018-10-03T21:32:50Z

@pglez82
Updated the PR with some additional logic

To reproduce, when uploading a large file, cancel the upload then edit the 'resume_upload' to have contents similar to the following:

{"error":{"code":"UnknownError","innerError":{"date":"2018-10-10T21:03:19.61Z","request-id":"1e95dd5f-9be3-4b64-9d64-b0811ad18ab2"},"message":""},"localPath":".\/DVDFab9210.exe"}

On restarting the application, the resume_file will be correctly validated.

Updated output when using the updated PR:

Loading config ...
Using Config Dir: /home/user/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/user/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: <redacted>
Default Root ID: <redacted>
Remaining Free Space: 112742891520
Fetching details for OneDrive Root
OneDrive Root exists in the database
Trying to restore the upload session ...
Restore file upload session failed - cleaning up session resume    <-- session upload file issue & cleanup
Syncing changes from OneDrive ...
Applying changes of Path ID: <redacted>
Uploading differences of .
Processing root
The directory has not changed
Uploading new items of .
Uploading file ./DVDFab9210.exe ...

…) (#176) * Handle an invalid response on session resume when a 4xx / 5xx response is generated from the OneDrive service

abraunegg · 2018-10-03T23:39:14Z

PR #176 merged

lock · 2019-01-05T22:35:37Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

abraunegg added Bug Something isn't working Investigating labels Sep 14, 2018

abraunegg added PR Provided - Needs Testing Waiting on Response and removed Investigating labels Sep 14, 2018

abraunegg added this to the 2.1.3 milestone Oct 3, 2018

abraunegg added a commit that referenced this issue Oct 3, 2018

update fix for issue #174

e06467c

* Update fix for issue #174

abraunegg added Fixed and removed PR Provided - Needs Testing Waiting on Response labels Oct 3, 2018

abraunegg added a commit that referenced this issue Oct 3, 2018

Resolve Key not found: expirationDateTime on session resume (Issue #174…

74b9163

…) (#176) * Handle an invalid response on session resume when a 4xx / 5xx response is generated from the OneDrive service

abraunegg closed this as completed Oct 3, 2018

lock bot locked and limited conversation to collaborators Jan 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

std.json.JSONException@std/json.d(508): Key not found: expirationDateTime #174

std.json.JSONException@std/json.d(508): Key not found: expirationDateTime #174

fgauvrit commented Sep 14, 2018

abraunegg commented Sep 14, 2018

abraunegg commented Sep 14, 2018

abraunegg commented Sep 20, 2018

pglez82 commented Oct 3, 2018

pglez82 commented Oct 3, 2018 •

edited by abraunegg

Loading

abraunegg commented Oct 3, 2018

pglez82 commented Oct 3, 2018

abraunegg commented Oct 3, 2018

pglez82 commented Oct 3, 2018

abraunegg commented Oct 3, 2018 •

edited

Loading

abraunegg commented Oct 3, 2018

lock bot commented Jan 5, 2019

std.json.JSONException@std/json.d(508): Key not found: expirationDateTime #174

std.json.JSONException@std/json.d(508): Key not found: expirationDateTime #174

Comments

fgauvrit commented Sep 14, 2018

Bug Report Details

abraunegg commented Sep 14, 2018

abraunegg commented Sep 14, 2018

abraunegg commented Sep 20, 2018

pglez82 commented Oct 3, 2018

pglez82 commented Oct 3, 2018 • edited by abraunegg Loading

abraunegg commented Oct 3, 2018

pglez82 commented Oct 3, 2018

abraunegg commented Oct 3, 2018

pglez82 commented Oct 3, 2018

abraunegg commented Oct 3, 2018 • edited Loading

abraunegg commented Oct 3, 2018

lock bot commented Jan 5, 2019

pglez82 commented Oct 3, 2018 •

edited by abraunegg

Loading

abraunegg commented Oct 3, 2018 •

edited

Loading