Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Synchronization fails with 'Key not found: parentReference' / 'Key not found: id' when syncing Shared Folders #144

Closed
lepennec opened this issue Aug 26, 2018 · 14 comments
Closed

Synchronization fails with 'Key not found: parentReference' / 'Key not found: id' when syncing Shared Folders #144

lepennec opened this issue Aug 26, 2018 · 14 comments
Assignees
@abraunegg
Labels
Bug Something isn't working Fixed
Milestone
2.1.4

Comments

@lepennec
Copy link

Bug Report Details

Describe the bug
When I try to sync my account, onedrive fails after a while with an error:
std.json.JSONException@std/json.d(494): Key not found: parentReference

Application and Operating System Details:

  • OS: Linux localhost.localdomain 4.17.14-202.fc28.x86_64 Fix config folder. #1 SMP Wed Aug 15 12:29:25 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Application version: onedrive v2.1.1-5-gbdc33e6
  • OneDrive Account Type: Personal
  • DMD64 D Compiler v2.080.0

To Reproduce

  1. Launch onedrive with onedrive --synchronize
  2. Wait for the following error
Loading config ...
Using Config Dir: /home/lepennec/.config/onedrive
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/lepennec/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: e251f8ca21dbd199
Default Root ID: E251F8CA21DBD199!103
Remaining Free Space: 1098435483478
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: E251F8CA21DBD199!103
Applying changes of Path ID: 184D10E6ED701165!62297
Key not found: parentReference
Syncing changes from OneDrive ...
Applying changes of Path ID: E251F8CA21DBD199!103
Applying changes of Path ID: 184D10E6ED701165!62297
Key not found: parentReference
Syncing changes from OneDrive ...
Applying changes of Path ID: E251F8CA21DBD199!103
Applying changes of Path ID: 184D10E6ED701165!62297
std.json.JSONException@std/json.d(494): Key not found: parentReference
----------------
/usr/include/dmd/phobos/std/exception.d:515 pure @safe void std.exception.bailOut!(std.json.JSONException).bailOut(immutable(char)[], ulong, const(char[])) [0x58bf5b]
??:? pure @safe inout(std.json.JSONValue)* std.exception.enforce!(std.json.JSONException).enforce!(inout(std.json.JSONValue)*).enforce(inout(std.json.JSONValue)*, lazy const(char)[], immutable(char)[], ulong) [0x636ba6]
??:? inout pure ref @safe inout(std.json.JSONValue) std.json.JSONValue.opIndex(immutable(char)[]) [0x5e7165]
src/sync.d:44 bool sync.changeHasParentReferenceId(ref const(std.json.JSONValue)) [0x5b34c4]
src/sync.d:409 void sync.SyncEngine.applyDifferences(immutable(char)[], const(char)[]) [0x5b4d73]
src/sync.d:206 void sync.SyncEngine.applyDifferences() [0x5b42c3]
src/main.d:401 void main.performSync(sync.SyncEngine, immutable(char)[], bool, bool, bool) [0x599c66]
src/main.d:269 _Dmain [0x5994ef]

Note that I have tried to resync with --resync and also to start from scratch in a new sync directory.

Please let me know if I can help and thanks for your tremendous job.

Erwan
@abraunegg
Copy link
Owner

@lepennec
In this case, using --resync does nothing - all it does it remove the local database cache. It should not be used as a troubleshooting item.

It appears to be failing on testing if the item object being downloaded is a shared folder item. Unless there has been a OneDrive API change, this should be OK.

Can you provide some details around your OneDrive personal account configuration & if you have any shared folders & so on?

@lepennec
Copy link
Author

OK. I've got a personal OneDrive account with around 17 item that I share and 3 that are shared to me.
The synchronization was working until end of July (as far as I remember).
Do not hesitate to ask me for more details.

@abraunegg
Copy link
Owner

@lepennec
To assist with debugging this issue, as it potentially relates to shared folders, please can you rebuild your client using the following PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/145/head:pr145
git checkout pr145
make
make install

Once rebuilt, please rerun the client with the following:

onedrive --synchronize --verbose

If you can send the resulting logfile to support@mynas.com.au rather than posting here that would be greatly appreciated.

@abraunegg abraunegg changed the title Synchronization fails with key parentReference not found Synchronization fails with 'Key not found: parentReference' Aug 26, 2018
@abraunegg
Copy link
Owner

Thanks for the log file. Will communicate via email whilst investigating.

@abraunegg abraunegg added OneDrive Personal OneDrive API Blocker An API issue prevents further work at this point in time and removed Waiting on Response labels Aug 28, 2018
@abraunegg
Copy link
Owner

So it appears that the OneDrive API response has changed for shared folders - which results in this error.

Raised a question here OneDrive/onedrive-api-docs#907 to validate if this is a change or regression.

@abraunegg
Copy link
Owner

Confirmed OneDrive API bug that will currently break this client.

@abraunegg abraunegg changed the title Synchronization fails with 'Key not found: parentReference' Synchronization fails with 'Key not found: parentReference' (Sync of Shared Folders) Aug 31, 2018
@abraunegg
Copy link
Owner

@lepennec
There has been an API change that re-introduces the missing data, however now the sync breaks. Will have to update this client to work with the new API responses.

@abraunegg abraunegg added the In Progress Currently being worked on label Oct 9, 2018
@abraunegg abraunegg changed the title Synchronization fails with 'Key not found: parentReference' (Sync of Shared Folders) Synchronization fails with 'Key not found: parentReference' / 'Key not found: id' when syncing Shared Folders Oct 9, 2018
@lepennec
Copy link
Author

lepennec commented Oct 9, 2018 via email

@abraunegg
Copy link
Owner

@lepennec
Please can you test / try the following PR to resolve this issue:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/191/head:pr191
git checkout pr191
make
sudo make install

I have tested this locally and I am able to sync OneDrive Shared Folders from OneDrive Personal accounts again without issue.

@abraunegg abraunegg added PR Provided - Needs Testing and removed In Progress Currently being worked on OneDrive API Blocker An API issue prevents further work at this point in time OneDrive Personal labels Oct 10, 2018
@abraunegg abraunegg added this to the 2.1.4 milestone Oct 10, 2018
@abraunegg
Copy link
Owner

Local validation of issue resolution:

Loading config ...
Using Config Dir: /home/user/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/user/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: <redacted>
Default Root ID: <redacted>
Remaining Free Space: 5368689793
Fetching details for OneDrive Root
OneDrive Root does not exist in the database. We need to add it.
Added OneDrive Root to the local database
Syncing changes from OneDrive ...
Applying changes of Path ID: <redacted>
Downloading file Document1.docx ... done.
Creating directory: Documents
Creating directory: 4B
Creating directory: 3A
Downloading file 4B/asdfwqer.txt ... done.
Downloading file Documents/asdf.txt ... done.
Downloading file Documents/asdf2.xtx ... done.
Downloading file 4B/testadsfa.asdf ... done.
Downloading file 4B/Document1.docx ... done.
Applying changes of Path ID: BC7D88EC1F539DCF!107       <-- shared root path
Creating directory: 3A/subfolder1
Downloading file 3A/3A-doc-initial.docx ... done.
Downloading file 3A/Document1.docx ... done.
Downloading file 3A/subfolder1/asdf.txt ... done.
Creating directory: 3A/subfolder1/subfolder2
Creating directory: 3A/subfolder1/subfolder3
Downloading file 3A/subfolder1/subfolder2/sqwe.txt ... done.
Creating directory: 3A/subfolder1/subfolder3/subfolder4
Downloading file 3A/subfolder1/subfolder3/subfolder4/asdfwerqwe.txt ... done.
Downloading file 3A/subfolder1/subfolder3/subfolder4/newfile.txt ... done.
Downloading file 3A/subfolder1/subfolder3/subfolder4/newfile2.txt ... done.
Uploading differences of .
Processing root
The directory has not changed
Processing Documents
The directory has not changed
Processing asdf.txt
The file has not changed
Processing asdf2.xtx
The file has not changed
Processing Document1.docx
The file has not changed
Processing 4B
The directory has not changed
Processing testadsfa.asdf
The file has not changed
Processing Document1.docx
The file has not changed
Processing asdfwqer.txt
The file has not changed
Processing 3A
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A-doc-initial.docx
The file has not changed
Processing subfolder1
The directory has not changed
Processing subfolder2
The directory has not changed
Processing sqwe.txt
The file has not changed
Processing subfolder3
The directory has not changed
Processing subfolder4
The directory has not changed
Processing asdfwerqwe.txt
The file has not changed
Processing newfile.txt
The file has not changed
Processing newfile2.txt
The file has not changed
Processing asdf.txt
The file has not changed
Processing Document1.docx
The file has not changed
Uploading new items of .
Applying changes of Path ID: <redacted>
Applying changes of Path ID: BC7D88EC1F539DCF!107

@abraunegg abraunegg self-assigned this Oct 10, 2018
@lepennec
Copy link
Author

Looks like it also works for me! Thanks!

@abraunegg
Copy link
Owner

@lepennec
Thanks for the confirmation

abraunegg added a commit that referenced this issue Oct 10, 2018
@abraunegg
Resolve syncing of Shared Folders due to OneDrive API change - Issue #…
53b72e0 
…144 (#191)

* Update handling of shared folders and items which was previously broken to a OneDrive API change
* Extract syncFolderName from either local or remote folder - not just local folder.
@abraunegg
Copy link
Owner

PR #191 merged

@lock
Copy link

lock bot commented Jan 5, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Jan 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@abraunegg abraunegg

Labels
Bug Something isn't working Fixed
Projects
None yet
Milestone
2.1.4
Development

No branches or pull requests
2 participants
@abraunegg @lepennec