Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to create folders on shared OneDrive accounts #217

Closed
3 tasks done
ghost opened this issue Oct 31, 2018 · 19 comments
Closed
3 tasks done

Unable to create folders on shared OneDrive accounts #217

ghost opened this issue Oct 31, 2018 · 19 comments
Labels
Bug Something isn't working Fixed OneDrive Personal

Comments

@ghost
Copy link

ghost commented Oct 31, 2018

Bug Report Details

I have an account with a shared folder, the content of this folder is not downloaded. I have also some strange scenarios:

  1. With the shared folder empty in my disk, if i add a file to it. Then the logs shows:
Processing root
Uploading new items of .
Uploading file ./IT/demo2 ...Remaining free space: 1104879927828
Uploading file ./IT/demo ...Remaining free space: 1104879927828

But the changes are not in OneDrive.

  1. If i add a new file inside the shared folder from the OneDrive WebUI, that file is never downloaded.

So looks like for some reasons he cannot detect the items in the shared folder when i sync.
We started this conversation in this issue: skilion/onedrive#428

Application and Operating System Details:

  • OS: Linux luciernaga 4.18.0-10-generic Sync Issue: Key not found: fileSystemInfo when syncing shared folders #11-Ubuntu SMP Thu Oct 11 15:13:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux // System 76 Pop OS! 18.10
  • GUI Desktop Gnome 3
  • Application version: onedrive v2.1.4-6-geb2cbfd
  • OneDrive Account Type: Personal
  • DMD or LDC compiler version: DMD64 D Compiler v2.080.1 // DUB version 1.9.0, built on May 31 2018

To Reproduce
Steps to reproduce the behavior if not causing an application crash:

  1. onedrive -m
  2. Do the process to register the app in the account
  3. Stop the app (CTRL+C)
  4. systemctl --user start onedrive

If issue is replicated by a specific 'file' or 'path' please archive the file and path tree & email to support@mynas.com.au

Complete Verbose Log Output
A clear and full log of the problem when running the application in the following manner (ie, not in monitor mode):

Oct 31 11:36:46 luciernaga onedrive[2594]: Initializing the Synchronization Engine ...
Oct 31 11:36:49 luciernaga onedrive[2594]: Initializing monitor ...
Oct 31 11:36:49 luciernaga onedrive[2594]: OneDrive monitor interval (seconds): 45
Oct 31 11:37:34 luciernaga onedrive[2594]: Syncing changes from OneDrive ...
Oct 31 11:37:36 luciernaga onedrive[2594]: Creating directory: Pictures
Oct 31 11:37:36 luciernaga onedrive[2594]: Creating directory: Documents
Oct 31 11:37:38 luciernaga onedrive[2594]: Downloading file Getting started with OneDrive.pdf ... done.
Oct 31 11:37:38 luciernaga onedrive[2594]: Creating directory: backup_delegated_account
Oct 31 11:37:38 luciernaga onedrive[2594]: Creating directory: OneNote
Oct 31 11:37:38 luciernaga onedrive[2594]: Creating directory: IT
Oct 31 11:37:39 luciernaga onedrive[2594]: Downloading file a ... done.

Run the application in a separate terminal window or SSH session and provide the entire application output including the error & crash. When posing the logs, Please format log output to make it easier to read. See https://guides.github.com/features/mastering-markdown/ for more details.

Application Log Output:

Verbose console log output goes here

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Bug Report Checklist

  • Detailed description
  • Reproduction steps (if applicable)
  • Verbose Log Output
@abraunegg
Copy link
Owner

@ppadial
How did you install the compiler?

DMD or LDC compiler version: DMD64 D Compiler v2.080.1 // DUB version 1.9.0, built on May 31 2018

If you used the Ubuntu packages to install DMD, there has been many instances of issues of the application not working right due to the compiler from Ubuntu packages. Can you uninstall DMD and install using the method in the readme & recompile ?

Once done, can you please run all your tests without using monitor mode:

onedrive --synchronize --verbose

@ghost
Copy link
Author

ghost commented Nov 1, 2018

i installed the compiler using Snap, does it have the same issues?

@abraunegg
Copy link
Owner

Potentially - ensure you install compiler as per pre-req in readme

@abraunegg
Copy link
Owner

abraunegg commented Nov 1, 2018

@ppadial
Unable to reproduce using:

  • Ubuntu 18.10 x84_64
Linux ubuntu-x64-18 4.18.0-10-generic #11-Ubuntu SMP Thu Oct 11 15:13:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • dmd-2.082.1
  • OneDrive Personal Account
  • OneDrive Personal Shared Folder

Results:

  • Local files are uploaded without issue to OneDrive root folder & shared folder
  • Files uploaded via web interface are downloaded without issue from OneDrive root folder & shared folder

Please re-test via installing the dependencies as per readme and not via Ubuntu packages.

@abraunegg
Copy link
Owner

Log - Initial Sync

 ./onedrive --synchronize --verbose
Loading config ...
Using Config Dir: /root/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /root/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5368698119
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Creating directory: 3A                                                                 <- This is the shared folder
Downloading file Document1.docx ... done.
Applying changes of Path ID: BC7D88EC1F539DCF!107
Downloading file 3A/3A-doc-initial.docx ... done.
Downloading file 3A/Document1.docx ... done.
Creating directory: 3A/subfolder1
Downloading file 3A/Book1.xlsx ... done.
Creating directory: 3A/subfolder1/subfolder2
Creating directory: 3A/subfolder1/subfolder3
Downloading file 3A/subfolder1/asdf.txt ... done.
Downloading file 3A/subfolder1/subfolder2/sqwe.txt ... done.
Creating directory: 3A/subfolder1/subfolder3/subfolder4
Downloading file 3A/subfolder1/subfolder3/subfolder4/asdfwerqwe.txt ... done.
Downloading file 3A/subfolder1/subfolder3/subfolder4/newfile.txt ... done.
Downloading file 3A/subfolder1/subfolder3/subfolder4/newfile2.txt ... done.
Uploading differences of .
Processing root
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A-doc-initial.docx
The file has not changed
Processing subfolder1
The directory has not changed
Processing subfolder2
The directory has not changed
Processing sqwe.txt
The file has not changed
Processing subfolder3
The directory has not changed
Processing subfolder4
The directory has not changed
Processing asdfwerqwe.txt
The file has not changed
Processing newfile.txt
The file has not changed
Processing newfile2.txt
The file has not changed
Processing asdf.txt
The file has not changed
Processing Document1.docx
The file has not changed
Processing Book1.xlsx
The file has not changed
Processing Document1.docx
The file has not changed
Uploading new items of .
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107     <- This is the shared folder

Upload new file - non shared folder:

./onedrive --synchronize --verbose
Loading config ...
Using Config Dir: /root/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /root/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5368698119
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107
Uploading differences of .
Processing root
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A-doc-initial.docx
The file has not changed
Processing subfolder1
The directory has not changed
Processing subfolder2
The directory has not changed
Processing sqwe.txt
The file has not changed
Processing subfolder3
The directory has not changed
Processing subfolder4
The directory has not changed
Processing asdfwerqwe.txt
The file has not changed
Processing newfile.txt
The file has not changed
Processing newfile2.txt
The file has not changed
Processing asdf.txt
The file has not changed
Processing Document1.docx
The file has not changed
Processing Book1.xlsx
The file has not changed
Processing Document1.docx
The file has not changed
Uploading new items of .
Uploading file ./newfile ... done.
Remaining free space: 5368698102
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107

Upload new file - shared folder:

./onedrive --synchronize --verbose
Loading config ...
Using Config Dir: /root/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /root/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5368698102
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107
Uploading differences of .
Processing root
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A-doc-initial.docx
The file has not changed
Processing subfolder1
The directory has not changed
Processing subfolder2
The directory has not changed
Processing sqwe.txt
The file has not changed
Processing subfolder3
The directory has not changed
Processing subfolder4
The directory has not changed
Processing asdfwerqwe.txt
The file has not changed
Processing newfile.txt
The file has not changed
Processing newfile2.txt
The file has not changed
Processing asdf.txt
The file has not changed
Processing Document1.docx
The file has not changed
Processing Book1.xlsx
The file has not changed
Processing Document1.docx
The file has not changed
Processing newfile
The file has not changed
Uploading new items of .
Uploading file ./3A/qwerqwer ... done.
Remaining free space: 5368698081
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107

Download new file from shared folder:

./onedrive --synchronize --verbose
Loading config ...
Using Config Dir: /root/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /root/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5368698102
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107
Downloading file 3A/subfolder1/newfile-online.docx ... done.                             <- new file from shared oline folder
Uploading differences of .
Processing root
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A-doc-initial.docx
The file has not changed
Processing subfolder1
The directory has not changed
Processing subfolder2
The directory has not changed
Processing sqwe.txt
The file has not changed
Processing subfolder3
The directory has not changed
Processing subfolder4
The directory has not changed
Processing asdfwerqwe.txt
The file has not changed
Processing newfile.txt
The file has not changed
Processing newfile2.txt
The file has not changed
Processing asdf.txt
The file has not changed
Processing newfile-online.docx
The file has not changed
Processing Document1.docx
The file has not changed
Processing Book1.xlsx
The file has not changed
Processing qwerqwer
The file has not changed
Processing Document1.docx
The file has not changed
Processing newfile
The file has not changed
Uploading new items of .
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107

@ghost
Copy link
Author

ghost commented Nov 1, 2018

Hello again,

I have fixed my issue, but i think there's an issue in code or in a border use case. I will try to leave here how i solved it.
BTW, my workaround works with the package built using the snap D software (i tried installing it using the install.sh info in readme but didn't work).

  1. So, I basically "unshared all my folders" to the account B from the account A.
  2. Then i went One By One to the subfolders i want to share and share them again.
  3. After doing that, everything was working fine. previously i just shared the "top" folders with no granularity.

NOTE: if i share just the "top" folders, the content is not downloaded.

After doing that, everything was working fine. Not sure if is something isolated to my account, or something related to the amount of subfolders level i have (a lot). I think this because the only shared folder that was downloading the content was the only one which didn't have any subfolder inside, just files.

Thanks for all your support,

@ghost
Copy link
Author

ghost commented Nov 1, 2018

I have a reproducible issue now.

  1. Using Compiler installed with install.sh as suggested in the readme.
  2. Having an account A with a folder called workspace and shared with Edit permissions to an AccountB.
  3. AccountB add the folder to his OneDrive.

The folder is correctly mapped when start the app

(dmd-2.082.1)user@luciernaga:~/source/github.third/onedrivea$ onedrive --verbose --confdir="~/.config/onedrive/OnedrivePersonal" --synchronize --monitor
Loading config ...
Using Config Dir: /home/user/.config/onedrive/OnedrivePersonal
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/user/Cloud/OnedrivePersonal
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 9d7667c927570d06
Default Root ID: 9D7667C927570D06!101
Remaining Free Space: 1104879927828
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 9D7667C927570D06!101
Applying changes of Path ID: 27CF47E2358920D5!67093
Applying changes of Path ID: 27CF47E2358920D5!56633
Applying changes of Path ID: 27CF47E2358920D5!61392
Applying changes of Path ID: 27CF47E2358920D5!61391
Applying changes of Path ID: 27CF47E2358920D5!67193
Applying changes of Path ID: 27CF47E2358920D5!67195
Uploading differences of .
Processing root
Uploading new items of .
Applying changes of Path ID: 9D7667C927570D06!101
Applying changes of Path ID: 27CF47E2358920D5!67093
Applying changes of Path ID: 27CF47E2358920D5!56633
Applying changes of Path ID: 27CF47E2358920D5!61392
Applying changes of Path ID: 27CF47E2358920D5!61391
Applying changes of Path ID: 27CF47E2358920D5!67193
Applying changes of Path ID: 27CF47E2358920D5!67195
Initializing monitor ...
OneDrive monitor interval (seconds): 45
Monitor directory: .
Monitor directory: ./workspace

After that i create a directory called "Crash" inside workspace and this is the log:

OneDrive Client requested to create remote path: ./workspace/Crash
The requested directory to create was not found on OneDrive - creating remote directory: ./workspace/Crash
std.json.JSONException@std/json.d(246): JSONValue is not an object
----------------
/home/user/dlang/dmd-2.082.1/linux/bin64/../../src/phobos/std/exception.d:515 pure @safe void std.exception.bailOut!(std.json.JSONException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x74f829d7]
/home/user/dlang/dmd-2.082.1/linux/bin64/../../src/phobos/std/exception.d:436 pure @safe bool std.exception.enforce!(std.json.JSONException).enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x74f82951]
??:? inout pure ref @property inout(std.json.JSONValue[immutable(char)[]]) std.json.JSONValue.object() [0x74fda531]
src/sync.d:1523 void sync.SyncEngine.saveItem(std.json.JSONValue) [0x74fb1cc7]
src/sync.d:1223 void sync.SyncEngine.uploadCreateDir(const(immutable(char)[])) [0x74fb0560]
src/sync.d:1133 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x74fafd66]
src/sync.d:1138 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x74fafddd]
src/sync.d:1138 void sync.SyncEngine.uploadNewItems(immutable(char)[]) [0x74fafddd]
src/sync.d:851 void sync.SyncEngine.scanForDifferences(immutable(char)[]) [0x74faea51]
src/main.d:447 void main.performSyn

And the application crashes.

Note: I stopped the usage of shared folders i think it gives too much problems, so im using a regular account with all the content on it. Using sync_list file to specify what i want to sync since it looks working fine with my other account.

Thanks !

@abraunegg
Copy link
Owner

abraunegg commented Nov 1, 2018

@ppadial
Was the folder creation in a 'shared account' or in the normal account?
Able to reproduce this create folder issue locally as well

@abraunegg abraunegg changed the title Shared folder content not downloaded Unable to create folders on shared OneDrive accounts Nov 1, 2018
@abraunegg abraunegg added Bug Something isn't working In Progress Currently being worked on Investigating OneDrive API Blocker An API issue prevents further work at this point in time and removed Compiler Issue Local Environment Issue Cannot locally reproduce issue Unable to reproduce Waiting on Response In Progress Currently being worked on labels Nov 1, 2018
@abraunegg
Copy link
Owner

@ppadial
The folder creation is failing due to an API change - this was working without issue

< Cache-Control: private
< Transfer-Encoding: chunked
< Content-Type: application/json
< request-id: 67458240-d1b4-4e7a-9bc6-b66eb9203e39
< client-request-id: 67458240-d1b4-4e7a-9bc6-b66eb9203e39
< x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"Australia Southeast","Slice":"SliceC","Ring":"5","ScaleUnit":"001","Host":"AGSFE_IN_10","ADSiteName":"ASE"}}
< Duration: 207.4371
< Strict-Transport-Security: max-age=31536000
< Date: Thu, 01 Nov 2018 19:51:05 GMT
< 
* Connection #1 to host graph.microsoft.com left intact
The requested directory to create was not found on OneDrive - creating remote directory: ./3A/asdf
* Found bundle for host graph.microsoft.com: 0x55bd52d85600 [can pipeline]
* Re-using existing connection! (#1) with host graph.microsoft.com
* Connected to graph.microsoft.com (20.190.142.178) port 443 (#1)
> POST /v1.0/drives/66d53be8a5056eca/items/66D53BE8A5056ECA!2481/children HTTP/1.1
Host: graph.microsoft.com
User-Agent: Phobos-std.net.curl/2.082 (libcurl/7.61.0)
Accept: */*
Content-Type: application/json
Authorization: bearer <redacted>
Content-Length: 27

* We are completely uploaded and fine
< HTTP/1.1 501 Not Implemented
< Cache-Control: private
< Transfer-Encoding: chunked
< Content-Type: application/json
< request-id: 5467b654-afa7-4b63-870b-74a466dcf43d
< client-request-id: 5467b654-afa7-4b63-870b-74a466dcf43d
< x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"Australia Southeast","Slice":"SliceC","Ring":"5","ScaleUnit":"001","Host":"AGSFE_IN_10","ADSiteName":"ASE"}}
< Duration: 503.7228
< Strict-Transport-Security: max-age=31536000
< Date: Thu, 01 Nov 2018 19:51:06 GMT
< 
* Connection #1 to host graph.microsoft.com left intact

@abraunegg
Copy link
Owner

Opened API Issue here:

OneDrive/onedrive-api-docs#939

@abraunegg abraunegg added OneDrive Personal and removed Bug Something isn't working Investigating labels Nov 1, 2018
@ghost
Copy link
Author

ghost commented Nov 8, 2018

Great Job, thx :)

@julian-alarcon
Copy link

So, the issue is not an issue, according to OneDrive devs. In that case, is there anykind of workaround=

@abraunegg
Copy link
Owner

@julian-alarcon
This is / was a OneDrive API change which now requires application modification, which is currently in progress since yesterday.

@abraunegg abraunegg added Bug Something isn't working In Progress Currently being worked on and removed OneDrive API Blocker An API issue prevents further work at this point in time labels Nov 20, 2018
@abraunegg
Copy link
Owner

@ppadial, @julian-alarcon
Please can you test the following PR to resolve this issue:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/244/head:pr244
git checkout pr244
make
make install

@abraunegg abraunegg added PR Provided - Needs Testing and removed In Progress Currently being worked on labels Nov 20, 2018
@abraunegg
Copy link
Owner

@ppadial, @julian-alarcon
Any update in testing this PR?

@abraunegg
Copy link
Owner

Local validation of fix:

Loading config ...
Using Config Dir: /home/alex/.config/onedrive
No config file found, using defaults
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/alex/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5368709120
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107
Uploading differences of .
Processing root
The directory has not changed
Processing 3A
The directory has not changed
Processing 3A
The directory has not changed
Processing qwerqwer
The file has not changed
Processing newdir
The directory has not changed
Processing newfile
The file has not changed
Processing Document1.docx
The file has not changed
Processing newdir2
The directory has not changed
Processing newfile
The file has not changed
Processing newdir3
The directory has not changed
Processing newfile
The file has not changed
Processing asadfasdf
The directory has not changed
Processing newfileadf
The file has not changed
Uploading new items of .
OneDrive Client requested to create remote path: ./3A/mynewdir
The requested directory to create was not found on OneDrive - creating remote directory: ./3A/mynewdir
Successfully created the remote directory ./3A/mynewdir on OneDrive
Uploading file ./3A/mynewdir/mynewfile ... done.
Remaining free space: 5368709099
Applying changes of Path ID: 66D53BE8A5056ECA!101
Applying changes of Path ID: BC7D88EC1F539DCF!107

abraunegg added a commit that referenced this issue Nov 21, 2018
) (#244)

* Update logic to use remote item details for folder creation
* Backport sync.d changes from PR #246 to this PR before merge
@abraunegg
Copy link
Owner

PR #244 merged

@ghost
Copy link
Author

ghost commented Nov 21, 2018

Ey sorry, i was quite busy this 2 days, i will check it tomorrow. Thanks for the support.
BTW, looks like shared folders api gives issues: MS do lots of changes here, so i'd go for non-shared folders solution.

@lock
Copy link

lock bot commented Jan 5, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Jan 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bug Something isn't working Fixed OneDrive Personal
Projects
None yet
Development

No branches or pull requests

2 participants