Use Microsoft Graph API and get rid of not-supported onedrivesdk #26

derrix060 · 2018-10-07T18:40:30Z

The current code is using a mix of old MS APIs.

This repo could authenticate using the new api, so it's something doable.

derrix060 · 2018-10-13T22:55:39Z

Instead of re-write all the onedrive sdk and use it only on this project, I decided to fork the oficial sdk and make the changes there. Feel free to help me on this: https://github.com/OneDriveSDK/onedrive-sdk-python

modelmat · 2018-10-14T01:04:34Z

I'll help where I can, but I'm back to school next week so I won't do much.

derrix060 · 2018-11-21T21:45:01Z

I was working on it yesterday, and almost all code changes are ready. I already get some information from Graph API in the code, I just need to make sure I can get everything.

derrix060 · 2018-11-27T08:45:59Z

known issues:

Also needs to support old accounts that were using onedrivesdk
I had to tweak a little bit onedrivesdk, so I need to put the changes here and find a good way to install this custom package
Subscriptions are not supported because Graph requires to have a HTTPS connection to the redirect_url. Testing with the default (NGROK) configuration didn't work.
Maybe there will be some api calls that I couldn't test yet that will fail.

modelmat · 2018-11-27T10:12:31Z

Since onedrivesdk is now graph-sdk-python, should we rename it throughout that repo and for the pypi package?

Also I'm not sure if this is relevant, but should we maybe call OneDriveSDK organisation something else and have onedriveClient in that organisation?

modelmat · 2018-11-27T10:15:59Z

Additionally, with the HTTPS connection we could:
a) Use #4 (comment)
or,
b) Serve up a temporary local web server which can handle the token requests (this should also stop token input errors) > there is an example given here

derrix060 · 2018-11-28T01:33:12Z

Since onedrivesdk is now graph-sdk-python, should we rename it throughout that repo and for the pypi package?

I've already created an issue to rename to graph-sdk-python, feel free to work on that. We still need to update the pypi package

Also I'm not sure if this is relevant, but should we maybe call OneDriveSDK organisation something else and have onedriveClient in that organisation?

I liked the idea to move onedriveClient to the organization and I'm okay renaming it... Any suggestion for the "new" name?

Additionally, with the HTTPS connection we could:
a) Use #4 (comment)

This wouldn't change the problem because the problem is not to have the token, but is to receive the notification from the subscription. I've reopened the issue anyway.

b) Serve up a temporary local web server which can handle the token requests (this should also stop token input errors) > there is an example given here

That's the idea of NGROK. I got 404 when trying to access the link...

modelmat · 2018-11-28T09:50:07Z

Oh, okay. That seems strange. I might take a look sometime, if the changes ever get pushed to the repo (or a branch on it).

derrix060 · 2018-11-28T10:02:15Z

I've pushed the changes here

derrix060 · 2018-12-03T05:51:33Z

Working fine locally. I've also tested with docker and is okay.

The key is to install the package using:

pip install .
pip install graph-sdk-python/

Please make sure to use a non-existent directory at this point, to avoid any conflict with the files.

@modelmat can you double check to see if it's working, and if so, I'll submit a PR.

modelmat · 2018-12-03T05:52:44Z

Ok, on it. Does this have tests?

derrix060 · 2018-12-03T05:54:24Z

Only the normal tests, the problem to test it, is I don't have a business account only for testing, and this won't be reliable at all.

modelmat · 2018-12-03T10:21:49Z

Okay, I was having issues with a keyerror, I believe it may be caused by a broken config - I'll try and delete it.

I got slightly sidetracked, so I'm not able to go any further at the moment. Sorry :P

modelmat · 2019-01-08T23:56:57Z

Starting onedrive-client ... OK
[2019-01-09 10:55:30,884] DEBUG: MainThread: Starting new HTTPS connection (1): login.microsoftonline.com
[2019-01-09 10:55:31,426] DEBUG: MainThread: https://login.microsoftonline.com:443 "POST /common/oauth2/v2.0/token HTTP/1.1" 200 3075
[2019-01-09 10:55:31,497] DEBUG: MainThread: Starting new HTTPS connection (1): graph.microsoft.com
[2019-01-09 10:55:32,634] DEBUG: MainThread: https://graph.microsoft.com:443 "GET /v1.0/me/drives HTTP/1.1" 200 None
[2019-01-09 10:55:32,638] DEBUG: MainThread: Refreshing repository session.
[2019-01-09 10:55:32,639] DEBUG: MainThread: Starting new HTTPS connection (1): login.microsoftonline.com
[2019-01-09 10:55:33,141] DEBUG: MainThread: https://login.microsoftonline.com:443 "POST /common/oauth2/v2.0/token HTTP/1.1" 200 3075
[2019-01-09 10:55:33,183] INFO: MainThread: Session for account 0137ANOXN6Y2GOVW7725BZO354PWSELRRZ will expire in 3599 seconds.
[2019-01-09 10:55:33,183] DEBUG: MainThread: Will refresh session in 3479 seconds.
[2019-01-09 10:55:33,184] INFO: MainThread: Sweeping onedrive_client temporary files from local repositories.
[2019-01-09 10:55:33,189] DEBUG: Worker-0: Started.
[2019-01-09 10:55:33,189] DEBUG: Worker-1: Started.
[2019-01-09 10:55:33,189] DEBUG: Worker-2: Started.
[2019-01-09 10:55:33,190] DEBUG: Worker-3: Started.
[2019-01-09 10:55:33,190] DEBUG: Worker-4: Started.
[2019-01-09 10:55:33,190] DEBUG: Worker-5: Started.
[2019-01-09 10:55:33,190] DEBUG: Worker-6: Started.
[2019-01-09 10:55:33,190] DEBUG: Worker-7: Started.
[2019-01-09 10:55:33,191] DEBUG: Worker-8: Started.
[2019-01-09 10:55:33,191] DEBUG: Worker-9: Started.
[2019-01-09 10:55:33,191] DEBUG: Worker-10: Started.
[2019-01-09 10:55:33,191] DEBUG: Worker-11: Started.
[2019-01-09 10:55:33,191] DEBUG: Worker-12: Started.
[2019-01-09 10:55:33,191] DEBUG: Worker-13: Started.
[2019-01-09 10:55:33,192] DEBUG: Worker-14: Started.
[2019-01-09 10:55:33,192] DEBUG: Worker-15: Started.
[2019-01-09 10:55:34,204] INFO: MainThread: Local ngrok API url: http://127.0.0.1:4040/api
[2019-01-09 10:55:34,216] DEBUG: MainThread: Starting new HTTP connection (1): 127.0.0.1
[2019-01-09 10:55:34,219] DEBUG: MainThread: http://127.0.0.1:4040 "GET /api/tunnels HTTP/1.1" 200 36
[2019-01-09 10:55:34,221] ERROR: MainThread: Error reading ngrok API: ngrok API did not return any tunnel.. Retry in 1sec.
[2019-01-09 10:55:35,227] DEBUG: MainThread: Starting new HTTP connection (1): 127.0.0.1
[2019-01-09 10:55:35,229] DEBUG: MainThread: http://127.0.0.1:4040 "GET /api/tunnels HTTP/1.1" 200 389
[2019-01-09 10:55:35,232] DEBUG: WebhookWorker: Started.
[2019-01-09 10:55:35,232] INFO: Webhook: Local webhook server listening on port 34263.
[2019-01-09 10:55:35,233] INFO: Webhook: Webhook server listening on https://88a8f1c8.ngrok.io/x1zIMAceBEG.
[2019-01-09 10:55:35,238] DEBUG: MainThread: Adding task StartRepositoryTask(drive=b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb)...
[2019-01-09 10:55:35,238] INFO: MainThread: Scheduled sync task for Drive b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb of account 0137ANOXN6Y2GOVW7725BZO354PWSELRRZ.
[2019-01-09 10:55:35,239] INFO: MainThread: Updating webhook for Drive b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb.
[2019-01-09 10:55:35,241] DEBUG: Worker-0: Got task StartRepositoryTask(drive=b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb).
[2019-01-09 10:55:35,242] DEBUG: MainThread: Starting new HTTPS connection (1): graph.microsoft.com
[2019-01-09 10:55:35,242] DEBUG: Worker-0: Adding task MergeDirectoryTask(/home/modelmat/OneDrive, deep=True, remote_unchanged=False, parent_remote_unchanged=False)...
[2019-01-09 10:55:35,243] DEBUG: Worker-0: Got task MergeDirectoryTask(/home/modelmat/OneDrive, deep=True, remote_unchanged=False, parent_remote_unchanged=False).
[2019-01-09 10:55:35,243] DEBUG: Worker-0: Removing watcher for "/home/modelmat/OneDrive"
[2019-01-09 10:55:35,249] DEBUG: Worker-0: Starting new HTTPS connection (1): graph.microsoft.com
[2019-01-09 10:55:35,701] DEBUG: Worker-0: https://graph.microsoft.com:443 "GET /v1.0/me/drives/b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb/root:/:/children HTTP/1.1" 400 None
[2019-01-09 10:55:35,703] ERROR: Worker-0: Encountered API Error: BadRequest - Resource not found for the segment 'root:'..
[2019-01-09 10:55:35,703] ERROR: Worker-0: Encountered API Error: BadRequest - Resource not found for the segment 'root:'.. Skip directory "".
[2019-01-09 10:55:35,798] DEBUG: MainThread: https://graph.microsoft.com:443 "POST /v1.0/me/drives/b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb/root:/:/subscriptions HTTP/1.1" 400 None
[2019-01-09 10:55:35,800] ERROR: MainThread: Error: BadRequest - Resource not found for the segment 'root:'.
[2019-01-09 10:55:35,802] WARNING: MainThread: Failed to create webhook. Will deep sync again in 21600 sec.
[2019-01-09 10:55:35,806] WARNING: MainThread: Executing <Handle gen_start_repo_tasks({'0137ANOXN6Y2...ZO354PWSELRRZ': [<onedrive_cli...x7f8039c1eda0>]}) at /usr/lib/python3.7/site-packages/onedriveClient-2.0.1-py3.7.egg/onedrive_client/od_main.py:129 created at /usr/lib/python3.7/site-packages/onedriveClient-2.0.1-py3.7.egg/onedrive_client/od_main.py:207> took 0.565 seconds
^C

finally got round to it, this error occurs.

modelmat · 2019-01-08T23:58:38Z

I just set this up after clearing all the config directories.
Output of onedrive-client-pref drives list:

 10:57:09  ~  onedrive-client-pref drive list
Reading drives information from OneDrive server...

All available Drives of authorized accounts:

  #  Account Email                       Drive ID                                                            Type      Endpoint                Quota
---  ----------------------------------  ------------------------------------------------------------------  --------  ----------------------  ------------------
  0  email@longdomaintoalignthetable.gg  b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb  business  Documents               9.9 GB/1.0 TB Used
  1  email@longdomaintoalignthetable.gg  b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbxl7fi0NSxASKw9r6p1z9IF  business  EduOneNoteAppDocuments  0.0 B/1.0 TB Used

Drives that have been set up:

 #0 - Drive "b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb":
   Account:     email@longdomaintoalignthetable.gg (0137ANOXN6Y2GOVW7725BZO354PWSELRRZ)
   Local root:  /home/modelmat/OneDrive
   Ignore file: /home/modelmat/.config/onedrive_client/ignore_v2.txt```

derrix060 · 2019-01-09T01:27:13Z

The above error is because was using a wrong backend.

So is it working now? Can I merge this -break- change?

modelmat · 2019-01-09T02:28:23Z

The log above was an error?
I don't think it is working - I couldn't find a for business command flag so I assumed both were the same?

derrix060 · 2019-01-09T19:46:42Z

@modelmat correct, both are the same.

I was referring to the longs that you put, more specifically to

[2019-01-09 10:55:35,798] DEBUG: MainThread: https://graph.microsoft.com:443 "POST /v1.0/me/drives/b!D7UiT0fOkUCFDKGEOWReIjCkcaYkNEpHrBCgM6IhYbwQZfhyXdaNSaKrLDD5UWXb/root:/:/subscriptions HTTP/1.1" 400 None
[2019-01-09 10:55:35,800] ERROR: MainThread: Error: BadRequest - Resource not found for the segment 'root:'.

modelmat · 2019-01-10T01:43:56Z

So, is this not an issue? So it's still syncing from this?

derrix060 changed the title ~~Use Microsoft Graph API~~ Use Microsoft Graph API and get rid of not-supported onedrivesdk Oct 13, 2018

derrix060 self-assigned this Oct 13, 2018

derrix060 added the enhancement label Oct 13, 2018

derrix060 mentioned this issue Nov 21, 2018

"No Drive associated with account" #32

Open

derrix060 added a commit that referenced this issue Nov 27, 2018

#26: use graph api for "business" and "personal" accounts

5a0c78e

derrix060 added a commit that referenced this issue Dec 3, 2018

#26: Add graph-sdk-python as submodule

4dc0f3a

derrix060 added a commit that referenced this issue Dec 3, 2018

#26: Use correct version from graph-sdk

b69567f

derrix060 added a commit that referenced this issue Dec 3, 2018

#26: use getpass to get username

c47a16c

derrix060 added a commit that referenced this issue Dec 3, 2018

Fix #26: update README to install graph-sdk from submodule

609cd52

derrix060 added a commit that referenced this issue Dec 22, 2018

#26: use https over ssh on gitmodule

fe45cbb

derrix060 added a commit that referenced this issue Dec 22, 2018

#26: specify branch on gitmodule

342ba0d

derrix060 added a commit that referenced this issue Dec 22, 2018

#26: get correct commit for submodule

59723ca

derrix060 added a commit that referenced this issue Dec 22, 2018

#26: install graph from submodule

38395ae

asppsa mentioned this issue Apr 16, 2019

Adding drives via "Sync" in SharePoint #48

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Microsoft Graph API and get rid of not-supported onedrivesdk #26

Use Microsoft Graph API and get rid of not-supported onedrivesdk #26

derrix060 commented Oct 7, 2018 •

edited

Loading

derrix060 commented Oct 13, 2018

modelmat commented Oct 14, 2018

derrix060 commented Nov 21, 2018

derrix060 commented Nov 27, 2018

modelmat commented Nov 27, 2018 •

edited

Loading

modelmat commented Nov 27, 2018 •

edited

Loading

derrix060 commented Nov 28, 2018 •

edited

Loading

modelmat commented Nov 28, 2018

derrix060 commented Nov 28, 2018

derrix060 commented Dec 3, 2018

modelmat commented Dec 3, 2018

derrix060 commented Dec 3, 2018

modelmat commented Dec 3, 2018

modelmat commented Jan 8, 2019

modelmat commented Jan 8, 2019 •

edited

Loading

derrix060 commented Jan 9, 2019

modelmat commented Jan 9, 2019

derrix060 commented Jan 9, 2019

modelmat commented Jan 10, 2019

Use Microsoft Graph API and get rid of not-supported onedrivesdk #26

Use Microsoft Graph API and get rid of not-supported onedrivesdk #26

Comments

derrix060 commented Oct 7, 2018 • edited Loading

derrix060 commented Oct 13, 2018

modelmat commented Oct 14, 2018

derrix060 commented Nov 21, 2018

derrix060 commented Nov 27, 2018

modelmat commented Nov 27, 2018 • edited Loading

modelmat commented Nov 27, 2018 • edited Loading

derrix060 commented Nov 28, 2018 • edited Loading

modelmat commented Nov 28, 2018

derrix060 commented Nov 28, 2018

derrix060 commented Dec 3, 2018

modelmat commented Dec 3, 2018

derrix060 commented Dec 3, 2018

modelmat commented Dec 3, 2018

modelmat commented Jan 8, 2019

modelmat commented Jan 8, 2019 • edited Loading

derrix060 commented Jan 9, 2019

modelmat commented Jan 9, 2019

derrix060 commented Jan 9, 2019

modelmat commented Jan 10, 2019

derrix060 commented Oct 7, 2018 •

edited

Loading

modelmat commented Nov 27, 2018 •

edited

Loading

modelmat commented Nov 27, 2018 •

edited

Loading

derrix060 commented Nov 28, 2018 •

edited

Loading

modelmat commented Jan 8, 2019 •

edited

Loading