Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Drive Support #654

Closed
Xuanwo opened this issue Sep 10, 2022 · 17 comments
Closed

Google Drive Support #654

Xuanwo opened this issue Sep 10, 2022 · 17 comments

Comments

@Xuanwo
Copy link
Member

Xuanwo commented Sep 10, 2022

Some notes about google drive API:

OpenDAL requires a path-based API, but google drive located files via its internal id. So we need to maintain:

  • a convert util function that converts a given path to id.
  • maintain an internal cache for the path -> id mapping, so we don't need to send requests every time.

API references: https://developers.google.com/drive/api/v3/reference

@Xuanwo
Copy link
Member Author

Xuanwo commented Sep 10, 2022

Cc @theshashankpal, I wrote some notes for you. Feel free to ask any questions here~

@Xuanwo Xuanwo moved this to 📋 Backlog in Xuanwo's Work Sep 15, 2022
@Xuanwo
Copy link
Member Author

Xuanwo commented Sep 27, 2022

Hi, @theshashankpal, are there any updates? What help can I give?

@imWildCat
Copy link
Contributor

I may take a look at either Dropbox/GDrive/OneDrive after finishing the list op of web dav.

I will choose the easiest one of these three first. Still learn Rust/OpenDAL in progress.

@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 17, 2023

I think Dropbox should be an easier entry point.

@Young-Flash
Copy link
Member

I will choose the easiest one of these three first. Still learn Rust/OpenDAL in progress.

hi, I also interested in this, which one would you chose? I may take another one to work on.

@imWildCat
Copy link
Contributor

@Young-Flash I'd like DropBox / OneDrive first

@Young-Flash
Copy link
Member

I'd like to work on google drive support for opendal and I have make some investigation, the basic idea here is to use google-drive3 dependency, which support google drive api v3 and under well maintenance. what do you think ? @Xuanwo

@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 26, 2023

the basic idea here is to use google-drive3 dependency, which support google drive api v3 and under well maintenance

Sorry, I have to say NO to this idea. OpenDAL prefers to maintain all HTTP based services ourselves.

Reasons:

  • OpenDAL only needs a subset of APIs like read/write/delete/... which are easy to implement.
  • OpenDAL provides well maintained HTTP infrastructure that is production ready and tested.
  • Avoid introducing new deps to reduce the extra maintenance work around licences, version bump, ...
  • And believe me, talking directly to the HTTP API is a lot easier and more fun than playing with the SDK 🤣

@Young-Flash
Copy link
Member

Young-Flash commented Feb 26, 2023

I see. and what about the path -> id cache? use other dep (like lru-cache) or do it ourselves (maybe a simple HashMap) ?

BTW, google drive access need oauth authorization, seems no easy to deal with it without extra deps.

@Xuanwo
Copy link
Member Author

Xuanwo commented Feb 26, 2023

use other dep (like lru-cache) or do it ourselves (maybe a simple HashMap) ?

Let's get started with a simple hashmap.

BTW, google drive access need oauth authorization, seems no easy to deal with it without extra deps.

Should be handled by reqsign. We can start with just accept a token file path.

@Young-Flash
Copy link
Member

I will take a look, but I may not get the work done soon cause I am struggling with my degree paper 😅

@imWildCat
Copy link
Contributor

@Young-Flash how's it going with your implementation? If not, I would pick this up in a few weeks.

@Young-Flash
Copy link
Member

I just submitted my degree thesis. I will work on this these days.

@imWildCat
Copy link
Contributor

@Young-Flash Thank you!

I will go for Dropbox first after OneDrive.

@Xuanwo
Copy link
Member Author

Xuanwo commented May 7, 2023

Basic support for gdrive has been added.

@Xuanwo Xuanwo closed this as completed May 7, 2023
@imWildCat
Copy link
Contributor

@Young-Flash would you like to implement the list operation of Google Drive? If not, I can take it.

@Young-Flash
Copy link
Member

@Young-Flash would you like to implement the list operation of Google Drive? If not, I can take it.

go ahead 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants