New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a one way sync option - so local files can never be deleted #1651

Closed
kaplandani opened this Issue Apr 7, 2014 · 24 comments

Comments

Projects
None yet
9 participants
@kaplandani

kaplandani commented Apr 7, 2014

This is crucial - since there are many posts about owncloud deleting files - Please add feature to totally block the client from deleting a file locally.

It is a good option for anyone who uses this as a backup solution and not more than that.

@flakrat

This comment has been minimized.

flakrat commented Apr 8, 2014

I'd like to plus 1 this request. For some strange reason OwnCloud client on my Windows box decided to delete my entire 6,000 song library without any explanation of why it decided to do it:

@dragotin

This comment has been minimized.

Contributor

dragotin commented Apr 8, 2014

@flakrat no nono, it's not working that way. Let's rather find out why that happened for you. Where was it deleted (local or remote), did you do any special, do you have any logs available, even if it's only the commonly written apache access_log? Have you used another software that uses the ownCloud WebDAV server or the local sync dir?
We have to either find out what lead to this, or find the bug. You can help!

@kaplandani

This comment has been minimized.

kaplandani commented Apr 8, 2014

@dragotin - This is a simple matter - without this feature, it is not SAFE to use this software.
I've deployed it, but having seen so many posts about deleted local files without recovery - I am afraid to use it anymore. I can't afford loosing files. So - it is one thing to find what causes file deletion, and another thing to add a safety mechanism that will prevents it from happening if the user don't want to and doesn't care loosing this feature (auto-delete)

I think a settings like "Approve all flle deletion" that the user can set, and a dialog that request confirmation on every local delete operation will solve this issue, and I think it is very simple to program. (finding all delete calls in the code and add this dialog before them if the global settings requires that.

I've seen too many threads in the forum and here regarding this to ignore this problem.

@dragotin dragotin added the Enhancement label Apr 8, 2014

@danimo

This comment has been minimized.

Contributor

danimo commented Apr 8, 2014

The matter is quite simple: If the software needs this mode to be useable, it's broken, and through this feature, it will never be fixed in crucial situations. It's a chicken and egg problem that we will never solve when implementing this feature.

So if you can't/won't help us resolving your specific situation, or you feel uneasy using ownCloud due to some gut feeling gained from reading the forums, as sad as it sounds, then don't.

That said we do take reports about data loss seriously and try to resolve them with the highest priority, as soon as we can find out what the problem is.

@danimo danimo closed this Apr 8, 2014

@dragotin

This comment has been minimized.

Contributor

dragotin commented Apr 8, 2014

And I also think we will rather think about the local trash bin option rather than not deleting anything locally. That would not have to do anything with syncing any more. See #528

@moscicki

This comment has been minimized.

Contributor

moscicki commented Apr 8, 2014

And I also think we will rather think about the local trash bin option rather than not deleting anything locally. That would not have to do anything with syncing any more.

I really think that an option to replace internally rm by mv to trahsbin (not necessarily the desktop trashbin) is really needed until the product get stable and rock solid.

As of now I know of two easy scenarios how to lose data on the client and there are probably more. I think we have to do everything possible to make sure that happens a user has an easy way to recover what they lost.

@dragotin

This comment has been minimized.

Contributor

dragotin commented Apr 8, 2014

@moscicki which two scenarios do you mean?

@moscicki

This comment has been minimized.

Contributor

moscicki commented Apr 8, 2014

@moscicki which two scenarios do you mean?

I think both of the scenarios have their issues in the tracker already. In short:

  1. Symlink scenario:
    mv DIR DIR.COPY
    ln -s DIR.COPY DIR
    sync
    this will delete the DIR on all connected sync clients

  2. Case scenario:
    linux:
    cp -a Dir dir
    sync
    macosx:
    sync
    this will delete Dir or dir back on the linux client -- not really sure what the outcome is

I have tried both with 1.5.3

Are you aware of more scenarios like that?

kuba

@danimo

This comment has been minimized.

Contributor

danimo commented Apr 8, 2014

@moscicki for recovery from trash, we have #1069. I also added a comment as to what the downsides are (i.e. this would still not be 100% reliable with long file names).

@danimo

This comment has been minimized.

Contributor

danimo commented Apr 8, 2014

Just for general information, we are aware of case sensitivity-related issues. They will be addressed in ownCloud 7 and ownCloud Client 1.7 respectively.

@kaplandani

This comment has been minimized.

kaplandani commented Apr 8, 2014

Move to a special folder is an option, but why will you not let the user
decide what the client course of action would be in case of delete ?

You can even add a buttons in the dialog: reject / approve / notify
owncloud on wrong delete scenario.

Which will send you the client log and user details for further
investigation.

A cloud software cannot cause damage to its users in ANY case.

The idea should be protecting data first then helping debug process...

Dani
On Apr 8, 2014 12:58 PM, "Daniel Molkentin" notifications@github.com
wrote:

Just for general information, we are aware of case sensitivity-related
issues. They will be addressed in ownCloud 7 and ownCloud Client 1.7
respectively.

Reply to this email directly or view it on GitHubhttps://github.com//issues/1651#issuecomment-39830587
.

@moscicki

This comment has been minimized.

Contributor

moscicki commented Apr 8, 2014

Interactive dialog for confirmation - could be an option. But the value of the sync client is that it works in the background. So maybe requiring attention of the user at every delete is a bit too much…

kuba

On Apr 8, 2014, at 12:27 PM, kaplandani notifications@github.com wrote:

Move to a special folder is an option, but why will you not let the user
decide what the client course of action would be in case of delete ?

You can even add a buttons in the dialog: reject / approve / notify
owncloud on wrong delete scenario.

Which will send you the client log and user details for further
investigation.

A cloud software cannot cause damage to its users in ANY case.

The idea should be protecting data first then helping debug process...

Dani
On Apr 8, 2014 12:58 PM, "Daniel Molkentin" notifications@github.com
wrote:

Just for general information, we are aware of case sensitivity-related
issues. They will be addressed in ownCloud 7 and ownCloud Client 1.7
respectively.

Reply to this email directly or view it on GitHubhttps://github.com//issues/1651#issuecomment-39830587
.


Reply to this email directly or view it on GitHub.

@dragotin

This comment has been minimized.

Contributor

dragotin commented Apr 8, 2014

@moscicki

  1. the symlink scenario is handled in #1299 which is fixed. I just verified exactly your scenario with the upcoming beta1 and it works as expected.

  2. Case scenario: Yes, known, and on the roadmap for the next large release. Your input is meanwhile appreciated on https://github.com/owncloud/core/wiki/Cross-Platform-File-Handling which outlines what we plan.

@kaplandani

This comment has been minimized.

kaplandani commented Apr 8, 2014

It should be optional.
Loosing data should not be an option.
Dani
On Apr 8, 2014 1:32 PM, "moscicki" notifications@github.com wrote:

Interactive dialog for confirmation - could be an option. But the value of
the sync client is that it works in the background. So maybe requiring
attention of the user at every delete is a bit too much...

kuba

On Apr 8, 2014, at 12:27 PM, kaplandani notifications@github.com wrote:

Move to a special folder is an option, but why will you not let the user
decide what the client course of action would be in case of delete ?

You can even add a buttons in the dialog: reject / approve / notify
owncloud on wrong delete scenario.

Which will send you the client log and user details for further
investigation.

A cloud software cannot cause damage to its users in ANY case.

The idea should be protecting data first then helping debug process...

Dani
On Apr 8, 2014 12:58 PM, "Daniel Molkentin" notifications@github.com
wrote:

Just for general information, we are aware of case sensitivity-related
issues. They will be addressed in ownCloud 7 and ownCloud Client 1.7
respectively.

Reply to this email directly or view it on GitHub<
https://github.com/owncloud/mirall/issues/1651#issuecomment-39830587>
.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHubhttps://github.com//issues/1651#issuecomment-39833045
.

@scolebrook

This comment has been minimized.

Contributor

scolebrook commented Apr 8, 2014

OC is not a backup destination. All data in any OC environment, or any other sync service, should be properly backed up. So an uncommanded delete in OC should never result in data loss.

A feature that prompts for confirmation of delete actions will be a significant disruption to users, especially users with lots of shared data. Considering a change by one user would result in a delete of the old file and a download of the new one for all other users sharing the file, this would be incredibly annoying in even a modestly sized environment. So people will turn it off, or not turn it on in the first place. And then we're right back where we started, with a bug that needs to be understood, isolated and fixed. So why not just fix it, rather than spend time applying a band aid that people will want to rip off.

@dragotin

This comment has been minimized.

Contributor

dragotin commented Apr 8, 2014

@scolebrook couldn't agree more, thanks.

@kaplandani

This comment has been minimized.

kaplandani commented Apr 8, 2014

Tell to people that lost their files.
I think OC is A great platform to create "live" backup. 99% of the time I
dont want to access the cloud. I just need to know that my files are safe.
My weekly backup will not help if the sync client destroyed my working
folder after two days work.

The prompt for delete as optional settings solves it to users like me and
can be ignored by users that highly collaborate on this cloud.

Also - using this feature will actually help detect this bug as some
deleted files will gone for weeks before noticed and here the "alarm" is
immediate.

Just my opinion

Dani
On Apr 8, 2014 5:36 PM, "Stephen Colebrook" notifications@github.com
wrote:

OC is not a backup destination. All data in any OC environment, or any
other sync service, should be properly backed up. So an uncommanded delete
in OC should never result in data loss.

A feature that prompts for confirmation of delete actions will be a
significant disruption to users, especially users with lots of shared data.
Considering a change by one user would result in a delete of the old file
and a download of the new one for all other users sharing the file, this
would be incredibly annoying in even a modestly sized environment. So
people will turn it off, or not turn it on in the first place. And then
we're right back where we started, with a bug that needs to be understood,
isolated and fixed. So why not just fix it, rather than spend time applying
a band aid that people will want to rip off.

Reply to this email directly or view it on GitHubhttps://github.com//issues/1651#issuecomment-39855375
.

@scolebrook

This comment has been minimized.

Contributor

scolebrook commented Apr 8, 2014

You've just described why your backup strategy is ineffective and leads to data loss. I would be fired if I thought once a week was an effective backup strategy for any product or use case I can possibly think up.

Your data loss issue is not because of ownCloud but because of infrequent backup. A sync service is not a backup. File system corruption damages a file and the damage is synced to all clients. Where's the backup?

3, 2, 1. The golden rule of backup. 3 copies, 2 local, 1 off site. Minimum, or you can't consider it backed up. I have 7 copies of every thing in my OC. The OC server, the daily server backup, my three computers, their backups (2 hourly, one daily, counted as 1 as they are all on the same drive) and my Crashplan backup (hourly). My total cost is a hamburger a month to Crashplan and $100 once for a 2TB hard drive for the local backups. I value my data much more highly than a hamburger a month but I think 7 copies (9 if you count the 3 backups on the one drive) is enough when 3 will do.

This isn't an issue about data loss as OC can never cause that in a system with a basic, effective backup strategy. It's an issue about uncommanded actions on files. A bug. The bug needs to be fixed. Having an annoying dialog interrupt users every few minutes as shared content is changed is bad UI even if there is no bug and these are legitimate deletions. It would be a tremendous waste of time implementing bad UI instead of fixing a bug. If there was no bug causing files to be deleted, would this dialog be helpful? No. It would be incredibly painful. So don't waste time on it. Treat the cause not the symptom.

@flakrat

This comment has been minimized.

flakrat commented Apr 8, 2014

@dragotin I think I may see what happened. I had set up OwnCloud server on a linux host, and OwnCloud client on my Win8 workstation. I'd started the first sync. The OwnCloud web client kept prompting that the PHP version was out dated, so I decided to reinstall the server with a more recent version of Ubuntu. During that process I decided to rebuild my data drive as a mirror instead of a logical volume spanning both drives.

This resulted in deleting to previous contents. After reinstalling OwnCloud, I restarted my OwnCloud client. I'm guessing it must have seen that the music files were no longer on the server and erased them and what I should have done was completely uninstall the client, delete any local configs, reinstall and start fresh.

I'll try and upload the logs when I get home tonight.

@dragotin

This comment has been minimized.

Contributor

dragotin commented Apr 9, 2014

@flakrat thanks for letting us know, that helps. But that does not sound like a client bug ;-)

@TheGrave

This comment has been minimized.

TheGrave commented Feb 10, 2016

There is another good reason to implement one-way sync which scares the shit out of me: imagine somebody hacks your server and purges the files. No matter on how many clients you have them synced if they are deleted transparently it would be a disaster. So my suggestion is:

  1. Non-existing files to be downloaded transparently on the owncloud client machine (as they are now).
  2. Modified files to be updated transparently (as they are now). The versions app should help you restore an older version of something is screwed.
  3. File deletion on client machine to be confirmed. Not by default but the user should have a checkbox in the config GUI to enable this option. Implementation can (soft of) follow Microsoft's multi-stage prompts on file deletion - first it asks you whether you want to delete the main folder, then sub-folders, then individual files. Without this being a configurable prompt it won't work for everybody - a lot of people are scripting data migrations and they need silent acceptance of all changes on clients.
@wrapper

This comment has been minimized.

wrapper commented Feb 17, 2016

+1 to this issue!

We are hosting our owncloud data folder in an encrypted container. Whenever the server is rebooted a passphrase must be entered to decrypt the data. OwnCloud detects this as the data being deleted and deletes all local data. Has happened twice so far!

If anyone has a workaround please let me know

@rfaelens

This comment has been minimized.

rfaelens commented May 31, 2016

@TheGrave: This should be covered by off-site backups, not by owncloud.
@wrapper: Do not start owncloud automatically. Start it manually after entering your encryption password. Or store the encryption password in your TPM chip and use a secure boot process (BIOS password, bootloader password, physical intrusion detection, etc).

@TheGrave

This comment has been minimized.

TheGrave commented May 31, 2016

Sorry but offsite backups do not make sense to me when data is already present in multiple locations. Why should I waste resources to handle another xTBs for data I already have in so many places?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment