Access to Team Drives (gdrive) #885
Comments
|
The Google Drive API doesn't work with Team Drive. I'm unaware of any public API that can interact with Team Drive. |
|
I'm told that the API is now enabled for Team Drives, but I've no idea what needs to be done to support it. |
|
@davidjgraph what happens if you use |
|
Hi |
|
This would be really nice to have, looking around it looks like the API supports team drives now. |
|
Looks like google.golang.org/api/drive/v2 supports Team Drives now as well. |
|
Does anyone want to work on this? I don't have access to a team drive unfortunately... |
|
@ncw Ok so it turns out you don't need to be part of a G Suite Business org to use Team Drives anymore. If you can send me your Google account email I can invite you to a Team Drive. |
|
@jahands I've sent you an email with my gmail! |
|
@ncw Thanks! You should now see Team Drives over in drive.google.com |
|
@jahands thanks for that - it is working fine! A bit of poking around discovers
How do you think this should be used from within rclone? Using the handy graphic above, rclone uses the contents of "My Drive" above. I could add a flag, say Or instead of a flag it could be an item in the config that you set the name of the team drive you want to use. I think this might make the most sense - it makes each team drive a different remote which would make it easy to copy between remotes. Alternatively I could make a kind of virtual file system making a Pseudo Directory "Team Drives" and within that a pesudo directory for each team drive the API lists. That goes against the way the drives are organised in reality (see pic above) which is bound to lead to trouble! Thoughts? |
|
I think Team Drives should be their own remote. They are meant to be
separate from your personal Drive files so it should be separate in the
config. I don't want Team Drives mixed with personal drives so I wouldn't
want them nested under My Drive.
…On Mon, Mar 13, 2017, 10:33 AM Nick Craig-Wood ***@***.***> wrote:
@jahands <https://github.com/jahands> thanks for that - it is working
fine!
A bit of poking around discovers
- I'll need to update the drive API package to get access to team
drives
- I'll need to list
<https://developers.google.com/drive/v2/reference/teamdrives/list> the
team drives to get their ids (which is also the id of their top level
folder)
- once I have their ids then rclone should work as normal (though
might need to set some extra flags in the files list method
<https://developers.google.com/drive/v2/reference/files/list>
How do you think this should be used from within rclone?
Using the handy graphic above, rclone uses the contents of "My Drive"
above. I could add a flag, say --drive-team-drive NAME which means that
it will use the team drive with that name instead.
Or instead of a flag it could be an item in the config that you set the
name of the team drive you want to use. I think this might make the most
sense - it makes each team drive a different remote which would make it
easy to copy between remotes.
Alternatively I could make a kind of virtual file system making a Pseudo
Directory "Team Drives" and within that a pesudo directory for each team
drive the API lists. That goes against the way the drives are organised in
reality (see pic above) which is bound to lead to trouble!
Thoughts?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#885 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AKOQXZauP7muspXB6OulO-kHHRp4xj9Aks5rlWGzgaJpZM4Ky3QD>
.
|
I like this. |
|
In terms of the process involved in adding a Team Drive, I think the best option would be to either offer it as an option within the creation of a gdrive remote, or have a separate remote. If it is done as a remote: If it is done as an extra step within a gdrive creation:
In terms of API implementation, it should be relatively simple: Taking a flag with the team drive name could be trickier than you think, as it doesn't appear to be supported in that way from the API. To get the IDs, then names of the Team Drive, you could, as @ncw suggested, list the team drives. The returned "items" could then be parsed into a name with another request for team drive metadata, which returns a "name" field, among others. All other modifications of files essentially just need a "teamDriveID" tacked onto it. Someone with familiarity with RClone's code could do this with relative simplicity. |
|
I'm not sure how well picking a name from a list would scale for admin users in orgs with a sizeable number of Team Drives (like one drive per department in a large corp or orgs with one Team Drive per project). This is an issue the official client also has, by the way, and I know for a fact that Google has received feedback about it (whether they'll ever tackle that is an entirely different question). |
|
@gustavorochakv good point! |
|
Another thing to possibly consider: If the destination/source is not a team drive, and the owner has read/write access to the team drive, then it should be server-side copy and move. A small example: |
|
Whenever I move something from Drive to a Team Drive, it says "migrating",
and generally takes some time to get there. So a server side move may take
hours depending how big it is. When these operations are done, it'd be a
good idea to tell the user it's "migrating" and may take a while, and don't
try again. If you try moving it a second time it gets messed up with
duplicates.
…On Thu, Apr 13, 2017 at 7:23 PM Michael Haas ***@***.***> wrote:
Another thing to possibly consider:
If the destination/source is not a team drive, and the owner has
read/write access to the team drive, then it should be server-side copy and
move.
A small example:
If I have source file A in a team drive, and a destination folder B,
between two remotes, then the following should happen:
file A should have a server-side copy made of file A, then the copy should
be moved (from the team drive into a private drive) in folder B.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#885 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AYvIjUtzWm3VEtdwYxh3AZN-KCYw39wpks5rvryQgaJpZM4Ky3QD>
.
|
|
Agreed, but on a related note, rclone server-side copy takes some time also. |
|
I've never found it to take hour for normal server-side copy though.
…On Fri, Apr 14, 2017, 11:00 AM Michael Haas ***@***.***> wrote:
Agreed, but on a related note, rclone server-side copy takes some time
also.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#885 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AYvIjSkwa-baAuDbapUDQZefrH26OhJuks5rv5gSgaJpZM4Ky3QD>
.
|
|
I have noted that appearance of files in a team drive is more of a visibility bug than it not being moved. It takes moments for it to copy, but I have noted that, while my main account couldn't see the file immediately, other accounts in the team drive could. From testing I have found it to take roughly the same time as a copy, if not a slightly longer amount of time. |
|
So is there support for team drives? I looked over this thread but didn't really get a clear yes/no/eta. |
|
Not currently.
…On Tue, May 23, 2017, 11:25 AM pmwoodward3 ***@***.***> wrote:
So is there support for team drives? I looked over this thread but didn't
really get a clear yes/no/eta.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#885 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AKOQXWs0SuOgVkkDPCqRze5USzXTfwz8ks5r8whigaJpZM4Ky3QD>
.
|
|
Shoot, any suggestions on how to backup a team drive locally? Pretty much my boss is wondering how we could get a copy of everything nightly. |
|
I got this error when I was configuring it: Seems to be working. I copied a folder to test things out. and one error was odd. Over and over it tried to copy 0B_raumxsbgcgYk9oOGhSSmZjcjg which I have no idea what that actually is..... I'm having a lot of failures though with that repeated message. Pretty wierd: |
|
I also still have the 404 File not found issue. Some files seem to work fine while others, especially media don't. None of my mp3s transfer while text files and images mostly work. I have no Idea what is causing that though. |
It's interesting, it does appear to be failing on media. But rclone size reports And browsing the uploaded files, I'm only really seeing documents, the occasional wav, but not the media. This upload us mostly media as we want to migrate a media share to a team drive. |
|
Hmm, something is not quite right. I haven't seen a specific problem with media but I have seen the uploads just disappearing. I'll investigate some more and post another beta. I must have missed something in the team drive docs... If someone could capture the http transaction which causes the file not found error that would be useful. Use -vv and --dump-bodies (or --dump-headers if that isn't practical) |
|
From what I can see, rclone has sanitised the debug output. dump-bodies put too much junk on my screen, probably because it was uploading a binary file, so I went with dump-headers The errors aren't consistent as to the filetype, it seems to happen more with larger files. Obviously media files are larger, so it happens more with those than small files. The particular directory I was trying to upload, having a look at what worked and didn't, anything over 8MB appears to have failed. |
|
Increasing |
|
Thanks @timwhite that was very helpful. I missed a teamdrive change in the upload chunked files code. This beta fixes that so it should fix the "File not Found" problem. https://pub.rclone.org/v1.36-156-gca76b3fb-team-drive%CE%B2/ |
|
@ncw Thanks. I'll give that a test and see how it goes. Will know by morning if I have lots of failures or not! |
|
@ncw Unfortunately didn't work. Here is a standard -vv https://pastebin.com/2cLGTtdg and here's with dump-headers https://cryptobin.co/39a667c6 - too big for pastebin, pw is 123 |
|
@Qwatuz I don't think you are using the latest version - can you try this one: https://pub.rclone.org/v1.36-156-gca76b3fb-team-drive%CE%B2/ (note that the 156 is the same, but the ID is different). |
|
@ncw Aha my bad! I will give it a try later today (someone will most likely beat me to it). ;) |
|
It worked fine for me! |
|
I can tell you this version is working fine with Team Drive |
|
@ncw after 10 hours of copying, it's transferred > 100GB with no errors! :-D Thanks for implementing Team Drives! |
|
Already transferred 300 GB onto a team drive, thank you @ncw . Even Media works now. Chunked Upload and big files work fine. Only issue left is that during first configuration it always states |
|
Excellent - thanks for testing everyone. I'll merge this to master so it will be in the general betas from now on. @gfrewqpoiu wrote
I've made a new issue about that here #1466 as it affects all the oauth remotes. The message is harmless though. The next beta will be here: https://beta.rclone.org/v1.36-158-ga5cfdfd2/ (uploaded in 15-30 mins). |
|
Hi Well done tested to migrate with success from file server Good job |
|
Successfully tested. My boss will be pretty freakin happy about this one. Thanks! |
ghost
mentioned this issue
|
@ncw So, if you have your main GDrive "gdriveMain" and your teamdrive "gdriveTeamDrive", if you do "rclone copy gdriveMain:/TestFile.txt gdriveTeamDrive:/" then it would attempt a server-side copy instead? As for testing whether it is possible, you could use the "capabilities.canCopy" from a Teamdrives.get request. Then, you could simply call a File.copy request of the file, with a "parents[]" section of the request body, simply containing the new destination. Unfortunately, I lack the knowledge of Go to implement this myself. |
|
Hello, I am trying to setup G Drive as a remote location by using Service Account. However when trying to access Team Drivers I get following error: Failed to configure team drive: list team drives failed: googleapi: Error 500: Internal Error, internalError Does SA work with Team Drives or should I use personal access token in this case? |
|
I set up a new team endpoint the other day and ran into this. Give it 5-10 minutes after setting up the new endpoint and try again. |
|
Thanks @gimpbully I will try again :) But in theory using Service Account should work also with Team Drives? |
I can't seem to figure out how to access a newly created Team Drive (fairly new feature, detailed briefly here under "Drive for Teams" https://butt.googleblog.com/2016/09/intelligent-tools-built-for-teams.html)
Team drive name is "HPCS" (not listed in lsd output, doesn't seem to be a way to configure it as a distinct config item).
root@rclone:/root# rclone lsd gdrive-hpcs:
-1 2016-11-09 20:34:32 -1 crypt
-1 2016-10-27 20:01:38 -1 bluearc
-1 2016-06-20 15:27:40 -1 HPCS-Backups
-1 2015-06-03 20:33:49 -1 2015-05
-1 2015-04-28 16:57:34 -1 Lucidchart
-1 2015-03-19 16:06:46 -1 BRC_Storage
2016/11/15 18:26:24
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 900ms
root@rclone:~#
The text was updated successfully, but these errors were encountered: