New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement cleanup for other remotes #575

Closed
ncw opened this Issue Jul 9, 2016 · 12 comments

Comments

Projects
None yet
6 participants
@ncw
Owner

ncw commented Jul 9, 2016

Implement cleanup for remotes

  • amazonclouddrive - not possible
  • b2 - done in #462
  • box - not possible easily (see below)
  • drive
  • dropbox - not possible
  • googlecloudstorage - not possible
  • hubic - not possible
  • local - not possible
  • onedrive - not possible
  • s3 - not possible
  • swift - not possible
  • yandex

@ncw ncw added this to the Soon milestone Jul 9, 2016

@cederberg

This comment has been minimized.

Show comment
Hide comment
@cederberg

cederberg Dec 2, 2016

Contributor

Trying v1.34 of rclone, I can't seem to get rclone cleanup to work properly against B2. The files remain listed (hidden) in the Backblaze file browser tool (web site).

Strangely, the hidden files do not show up when running rclone cleanup -v .... A guess might be that this is the root cause. Also tried with the --b2-versions flag, but to no avail.

Contributor

cederberg commented Dec 2, 2016

Trying v1.34 of rclone, I can't seem to get rclone cleanup to work properly against B2. The files remain listed (hidden) in the Backblaze file browser tool (web site).

Strangely, the hidden files do not show up when running rclone cleanup -v .... A guess might be that this is the root cause. Also tried with the --b2-versions flag, but to no avail.

@ncw

This comment has been minimized.

Show comment
Hide comment
@ncw

ncw Dec 2, 2016

Owner

@cederberg rclone cleanup should work properly with b2 now. Can you open a new issue with a log file and some instructions on how to reproduce please? I thought this might be related to #604 but I fixed that apparently!

Owner

ncw commented Dec 2, 2016

@cederberg rclone cleanup should work properly with b2 now. Can you open a new issue with a log file and some instructions on how to reproduce please? I thought this might be related to #604 but I fixed that apparently!

@cederberg

This comment has been minimized.

Show comment
Hide comment
@cederberg

cederberg Dec 2, 2016

Contributor

Hmm. Tried to reproduce, but now it works. Weird. The B2 web UI is sometimes slow, but shouldn't take several minutes to update... Hmm...

Ignore this for now. I'll file a proper bug if I see this again (and can repro the issue). Thanks for the quick support!

Contributor

cederberg commented Dec 2, 2016

Hmm. Tried to reproduce, but now it works. Weird. The B2 web UI is sometimes slow, but shouldn't take several minutes to update... Hmm...

Ignore this for now. I'll file a proper bug if I see this again (and can repro the issue). Thanks for the quick support!

@ncw

This comment has been minimized.

Show comment
Hide comment
@ncw

ncw Dec 2, 2016

Owner

@cederberg might be caching I suppose... File a bug if you can reproduce it! Thanks, Nick

Owner

ncw commented Dec 2, 2016

@cederberg might be caching I suppose... File a bug if you can reproduce it! Thanks, Nick

@eharris

This comment has been minimized.

Show comment
Hide comment
@eharris

eharris Dec 30, 2016

I haven't had a chance to dig into the code yet, but I'm wondering if cleanup is smart enough to only "cleanup" the files that rclone deleted, or if it does a whole-storage empty/purge? If the latter, I'm not sure I'd be comfortable with that without a pretty clear confirmation or --yes-really kind of option.

eharris commented Dec 30, 2016

I haven't had a chance to dig into the code yet, but I'm wondering if cleanup is smart enough to only "cleanup" the files that rclone deleted, or if it does a whole-storage empty/purge? If the latter, I'm not sure I'd be comfortable with that without a pretty clear confirmation or --yes-really kind of option.

@ncw

This comment has been minimized.

Show comment
Hide comment
@ncw

ncw Dec 31, 2016

Owner

@eharris cleanup will do its stuff on all files, regardless of whether rclone deleted them. It should only remove already deleted files.

If the latter, I'm not sure I'd be comfortable with that without a pretty clear confirmation or --yes-really kind of option.

Lots of rclone commands can cause data loss if used incorrectly. The --dry-run option is for those cases!

Owner

ncw commented Dec 31, 2016

@eharris cleanup will do its stuff on all files, regardless of whether rclone deleted them. It should only remove already deleted files.

If the latter, I'm not sure I'd be comfortable with that without a pretty clear confirmation or --yes-really kind of option.

Lots of rclone commands can cause data loss if used incorrectly. The --dry-run option is for those cases!

@timwhite

This comment has been minimized.

Show comment
Hide comment
@timwhite

timwhite Jun 18, 2017

Moving comments from #1489 to here:
It would be great to support Google Drive "Cleanup", or at least the ability to empty the Trash. Both V2 and V3 of the API support it https://developers.google.com/drive/v3/reference/files/emptyTrash

timwhite commented Jun 18, 2017

Moving comments from #1489 to here:
It would be great to support Google Drive "Cleanup", or at least the ability to empty the Trash. Both V2 and V3 of the API support it https://developers.google.com/drive/v3/reference/files/emptyTrash

@juanluisbaptiste

This comment has been minimized.

Show comment
Hide comment
@juanluisbaptiste

juanluisbaptiste Jul 25, 2017

Any ETA on google drive ?

juanluisbaptiste commented Jul 25, 2017

Any ETA on google drive ?

ishuah added a commit to ishuah/rclone that referenced this issue Sep 1, 2017

ishuah added a commit to ishuah/rclone that referenced this issue Sep 1, 2017

ishuah added a commit to ishuah/rclone that referenced this issue Sep 1, 2017

ncw added a commit that referenced this issue Sep 6, 2017

ishuah added a commit to ishuah/rclone that referenced this issue Sep 7, 2017

ishuah added a commit to ishuah/rclone that referenced this issue Sep 7, 2017

ishuah added a commit to ishuah/rclone that referenced this issue Sep 7, 2017

ishuah added a commit to ishuah/rclone that referenced this issue Sep 7, 2017

ncw added a commit that referenced this issue Sep 8, 2017

@ishuah

This comment has been minimized.

Show comment
Hide comment
@ishuah

ishuah Sep 17, 2017

Collaborator

@ncw I've been looking into cleanup for amazonclouddrive, box, dropbox and onedrive.

amazonclouddrive: There doesn't seem to be any documentation on permanently deleting trashed items with the Amazon Cloud Drive API. Docs

box: The Box API provides an endpoint for listing trashed items and permanently deleting trashed items. However, there's no endpoint for permanently deleting trashed items and the Box API does not support batch API calls. Cleanup is not feasible since it involves listing all trashed items and making a delete request for each item. Docs

dropbox: There doesn't seem to be any documentation on permanently deleting trashed items with the Dropbox API. From my understanding, it is not yet supported. Docs

onedrive: Same as dropbox and amazonclouddrive, the OneDrive API does not expose a way to permanently delete trashed items. Docs

Collaborator

ishuah commented Sep 17, 2017

@ncw I've been looking into cleanup for amazonclouddrive, box, dropbox and onedrive.

amazonclouddrive: There doesn't seem to be any documentation on permanently deleting trashed items with the Amazon Cloud Drive API. Docs

box: The Box API provides an endpoint for listing trashed items and permanently deleting trashed items. However, there's no endpoint for permanently deleting trashed items and the Box API does not support batch API calls. Cleanup is not feasible since it involves listing all trashed items and making a delete request for each item. Docs

dropbox: There doesn't seem to be any documentation on permanently deleting trashed items with the Dropbox API. From my understanding, it is not yet supported. Docs

onedrive: Same as dropbox and amazonclouddrive, the OneDrive API does not expose a way to permanently delete trashed items. Docs

@ncw ncw modified the milestones: Soon, v1.38 Sep 17, 2017

@ncw

This comment has been minimized.

Show comment
Hide comment
@ncw

ncw Sep 17, 2017

Owner

@ishuah - thanks for working on this ticket and the informative info above.

I make this ticket all done now - do you agree?

Owner

ncw commented Sep 17, 2017

@ishuah - thanks for working on this ticket and the informative info above.

I make this ticket all done now - do you agree?

@ishuah

This comment has been minimized.

Show comment
Hide comment
@ishuah

ishuah Sep 17, 2017

Collaborator

@ncw I agree, this ticket is done.

Collaborator

ishuah commented Sep 17, 2017

@ncw I agree, this ticket is done.

@ncw

This comment has been minimized.

Show comment
Hide comment
@ncw

ncw Sep 25, 2017

Owner

Excellent - thank you very much @ishuah

Owner

ncw commented Sep 25, 2017

Excellent - thank you very much @ishuah

@ncw ncw closed this Sep 25, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment