Improved Conflict Handling: Save Conflict Files to Server #4557
Comments
|
Hmm. 99.99% of all users seems to be happy with the current conflict files handling. |
|
Is there a way to globally check if there are unsynced conflict files? Other than asking every user individually for logfiles or local directory listing ;-) I think this would help to understand if there is a practical problem or not and maybe this could be a first step to tackling this issue. Would this information be stored in the syncdb journal file (in which case we could try to recall it to the server and analyze the whole bunch automatically)? I tried to count 412 PUT responses in our http log files but then I realized it is only a subset of conflict cases (and I do not have any feeling how representative this subset is). So I still do not have a solid data point to reason about this issue for our system. Dropbox does sync conflict files and probably there is a good reason to it. I am not sure if "99.99% of all users seems to be happy" is a good indicator in general because for example "99.99% of all users" did not notice the corruption problem that was actually a serious one (and statistically significant if you have above 10^7-10^8 files in the system). |
|
I like this idea but I have a concern in how the average, IT illiterate user, would be able to understand what's happening. If you can have two versions of the same file on two users computers with the same name, that's very bad. In the event of a conflict, I think a conflict file should be generated and the server copy sync down just as it is now (that tells the user something really, really needs their attention because they always take notice of notifications and messages from the ownCloud app). But the client syncs the conflict file back to the server as a branched version. This allows the switching of the branch easily. If the branch switches, then the local conflict file goes away as it then becomes a dupe. One edge case that would need to be considered. User 1 makes an edit that results in a conflict file. A new branch is created. The branch is switched server side so now the official version is the previously conflicted version. User 2 has been offline all this time but has edited the file. They've been working on the now non-current branch. What happens when they come back online. Does their current version sync to the non-current branch and then get replaced by the current branch of the file? How does the software inform User 2 of what has happened? Do the other users get informed that there's now an additional version in the other branch? Branching is a fairly easy concept to grasp as a developer but I think it's going to take a lot of explanation get that light bulb moment for many non technical users. This is not easy to do in user friendly way. It will take some special UI to make this intuitive. LibreOffice Online when they implement live collaborative editing + richdocuments is a better solution to this problem IMHO. Avoid the creation of conflicts entirely, then you don't have to deal with them, branching, etc. |
|
We are currently running into this problem. Maybe instead of doing this, let the user decide who wins, server or client. So simply said, let the user choose who wins, the server or the client instead of chosing it for them. |
|
@guruz @ogoffart Could we start with a lightweight approach? Just upload with filename and adding |
|
From an end user perspective, everyone needs/must see that there is a conflicted version. Otherwise, folks will continue to work on their version of the document, rather than get with other document users to resolve the issue. Agree with @michaelstingl.
Users like our organization, who have used Dropbox in the past, are very used to this happening and have internal processes for resolving these issues between user versions if they do happen.
Our organization is a relatively large workgroup and have never encountered this issue. I don't think we should guess what users must experience (99.99%). Instead, why not make this an admin decision? Perhaps make this a selection in the admin console, or if that's not feasible, as a client setting? |
|
Dropbox's approach is pretty straightforward: |
|
I can not really read from the dropbox text above if they have the conflict files all on the server. We did not do that initially as we did not want to spread all conflicts that somebody might create for whatever reason to the harddisks of each and every user of a share. That would probably also confuse people. I think the only good solution is to do it via the versions app. |
|
Yes, Dropbox puts all conflicted files on the server (see my original note). The files are marked with the username as well, which helps in resolving the issues (see attached image at link: [https://dl.dropboxusercontent.com/u/1595301/Capture.JPG] Think of the organizational disruption if you DON'T spread all conflicts to the server. First, they don't occur very often. When they do, vital information/revisions could be lost. It is better to have the issue of conflicts than not (again, Dropbox seems to have the best implementation). On an aside, the Dropbox client monitors if 2 or more users are in the same doc and warns if there's potential for conflict. |
|
The first function of a system that automatically handles files on behalf of the user is to protect the data. Local conflict file which is not uploaded to the server is a potential data loss. The larger the system the higher the probability of the data loss. I would NOT bundle it with the versions app. At least not at the level of sync protocol. A PUT of a conflict file should be marked as such (e.g. using an extra header) and the server implementation should be free to handle it appropriately (e.g. via versions app). I can imagine other implementations, not via a version app but on a separate list on the server. |
|
@moscicki ok, true comment of course. PUTting the conflicted file with an extra header makes sense, but would that PUT go against the original name, or using the usual conflict name with time stamp? |
|
Good question. I'd rather use the original name because then the server can
identify the source file and apply some logic based on that (e.g transform
the name into conflict file pattern or store as version). I also think
that it could be a better idea to make the server have the last say about
the timestamp it wants to use (not the client, to avoid any possibility of
clashes). But obviously that changes the semantics of PUT quite a bit
because the resource specified in the URL is not the same as the one
actually updated (not sure if there are any other side effects of that in
general HTTP world.). It would also make for the server mandatory to handle
the special header, else it would be uploaded as a genuine file version (is
this a problem for current server implementation, e.g. conflict loops seen
in the past?).
I would imagine something like that:
PUT filename.ext
OC-CONFLICT: YYYYMMDD-HHMM
Would result in, for example, storing:
filename_conflict-<YYYYMMDD>-<HHMM>-<USERNAME>.ext on the server
I would add USERNAME for conflicts in shared folders.
The question is: what to do if a user starts modifying the conflict file? I
think one should keep on uploading it. However, I would not download the
conflict files (the server could "hide" them from PROPFIND listing). In
this way we can avoid having conflicts of conflicts, loops etc.
Alternative of the above proposal is:
PUT filename_conflict-YYYYMMDD-HHMM-USERNAME.ext
OC-CONFLICT: filename.ext
In this one the server also has access to the original filename (what I
think is important).
…On Wed, Jul 12, 2017 at 11:03 AM, Klaas Freitag ***@***.***> wrote:
@moscicki <https://github.com/moscicki> ok, true comment of course.
PUTting the conflicted file with an extra header makes sense, but would
that PUT go against the original name, or using the usual conflict name
with time stamp?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4557 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAl9jTYJQ8pkFpzrvt4qGvmVOBAAr3ofks5sNIvygaJpZM4HslaE>
.
--
---
Best regards,
Kuba
|
|
Isn't it better to let the user decide what to do when a conflict happens?
Like what the QSync client does, it shows a notification and has 4 options (can't remember them precisely):
1: override local version with servers version
2: override server version with local version
3: generate conflict file (local version is changed into a conflict file)
4: do nothing, keep both
This way the user can choose what happens and I think this is better.
|
|
A user has a choice now: he can delete local conflict file (1), rename
local conflict file into original name (2), do nothing (3) and do nothing
(4).
A particular problem we are trying to solve in this ticket is that local
conflict file is not saved on the server so there is a potential data loss
if a user does nothing.
The sync client should take "data safe" decisions without user's attention
such as mandatory notification. Users usually do not pay attention to these
applets running in the background and also it is impractical/annoying if
notifications are popping up for a larger number of files. If, however,
such a notification was deemed nontheless useful it can be easily
implemented on top of proposed scheme as a convenience function on the
interface.
…On Sat, Jul 15, 2017 at 8:22 AM, dvdbot ***@***.***> wrote:
Isn't it better to let the user decide what to do when a conflict happens?
Like what the QSync client does, it shows a notification and has 4 options
(can't remember them precisely):
1: override local version with servers version
2: override server version with local version
3: generate conflict file (local version is changed into a conflict file)
4: do nothing, keep both
This way the user can choose what happens and I think this is better.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4557 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAl9jXauPISFtUHhKwFbcvkTwz1Xg4Vrks5sOFq5gaJpZM4HslaE>
.
--
---
Best regards,
Kuba
|
|
Agree with @moscicki, the client should protect the intregity of both the base file and ensure the org does not lose productivity from work completed in a conflicted file but not visible to coworkers and managers. I've used the dropbox applet that notifies of risk of conflict and 2 things:
|
|
Hello, Any news on this? kuba |
|
I agree with @crj2, any chance this feature might come available soon, this is the only issue preventing me from moving this onto a production machine. |
|
The current status is that the patch is merged and the behavior can be enabled though a server capability. |
|
@ckamm - Having the same issue here, running 2.4.0 build 8836 for mac. Do you know which release will hopefully cover this fix? Thank you for your support with this :) |
|
@julesmoretti The patch is available in 2.5.0 and onwards. This is currently controled by a server capability that has no configuration in the server yet. |
|
@ckamm also do you know this will also handle the And upon clicking on details: Issue Just to make sure :) |
|
@julesmoretti Yes, if conflict-uploading is enabled, the message will just speak about an unresolved conflict, and not say that the file was not uploaded. :) |
https://doc.owncloud.org/desktop/2.5/conflicts.html#uploading-conflicts-experimental |
|
I forgot to paste the download link to 2.5.0 beta1. |
|
Hm, the documentation lacks an explanation what you can and should expect when enabling that feature. So I doubt many people will test. |
linux mint taratestpilotcloud version 2.5.0daily20180808 (build 10016) Feature must be enabled through
Provoke a conflict by
BAD: replacing the file on the server does not cause a conflict, it explodes: #6701 PUT requests on a conflict should contain Tentative: OK: mitmproxy shows BAD: OC-ConflictBasePath is 'documented' in the issue, but missing. Maybe server would do nice versioning for that in the future? Both original file and conflict file are on the server, and Even older conflict files get uploaded once the upload conflicts feature is enabled BAD: username is missing in the conflict files that were generated before the feature is enabled. BAD: The tab 'not synced' now talks about a nicely synced (conflict) file. |
|
@dragotin people not testing stuff has many reasons. There is documentation, hooray!!!!. It could be worse!!! Actually I didn't read that snippet in the first place and discovered the environment variable in the source... enhancement request: That behaviour should be made default asap! It is a huge improevement to have the conflict files on the server. Even if the server does not do any extra magic right now. It is a great feature! |
|
@jnweiger: your test can easily be automated with smashbox and thus effortlessly rerun at will until the end of world. we can have a look at this together. |
|
@jnweiger in your testcase where you modify both on server and client (to actually create the conflict), which version is the file with the original file name after the whole conflict sync happened? Maybe that really should be changed now and make it the latest version, ie the one that was modified last? Is that what user expect? |
Can you please open an issue for that?
I suppose that's fine.
Right.... Although it is still nice to have this information there since we want to act upon this. @moscicki : We also have automated testing for this feature here: https://github.com/owncloud/client/blob/master/test/testsyncconflict.cpp But of course, this does not replace real testing on a real server. |
|
@dragotin Yes in all cases of conflict tested, the version from the server was kept in place, versions from the local client were renamed. Assuming nobody edits on the web interface, but all changes come through syncing clients, the client that syncs first would win. Which is the opposite of your suggestion "last client should win". A strong reason for "last edit wins" would be if we could assume that later authors have seen, considered and rejected (or merged) earlier changes. In our case they cannot see each other until after the conflict. The timestamp should be the modify timestamp of the file on the client. Not the moment when it got successfully synced to the server. Right? |
|
Enhancement idea: environment variable OWNCLOUD_UPLOAD_CONFLICT_FILES=1 should be a config file setting. |
|
Agreed, make this a config file setting and corresponding gui setting ZT |
|
There's a great little program called "allwaysync" that has a lovely conflict resolution tool. The key to the program's utility is that the conflict list is provided for in-program resolution. I have a folder with a lot of meta information modifications which generate conflicts with the server. The local files have the latest data, but they were all renamed. Now there's no straight-forward way to, say, select all of these conflicts and specify that the local file wins, please delete the copied down server file, un-rename the local file, and sync back to the server. That is, how about a UI tool that allows straight forward manual selection of which file should win and then executing that directive? |
|
It seems that if you rename the conflict file to overwrite the original, it then takes precedence over the server file and the next sync will overwrite the server without generating a conflict report.. So the problem of deconflicting client precedence is only to bulk rename all the conflict files overwriting the copies downloaded from the sever like: find -iname *_conflict-2016* -type f -exec rename -n -f 's/_conflict-2016[0-9]{4}-[0-9]{6}//' {} \;
as a dry run, then removing the "-n" flag to really do it like: find -iname *_conflict-2016* -type f -exec rename -f 's/_conflict-2016[0-9]{4}-[0-9]{6}//' {} \;
This will recurse through directories below where the command is executed, so be warned if some conflicts should be resolved with the server file taking precedence. Note that all my conflicts seem to have "2016" as the prefix, YMMV on that. If the years vary, something like find -iname *_conflict-* -type f -exec rename -n -f 's/_conflict-[0-9]{8}-[0-9]{6}//' {} \;
should do the trick, as long as "_conflict-" isn't used |
|
The worst thing in all this is, that there is really no SANE way to resolve the conflict. The only way which works is to open the server version file and the local one (renamed to conflict file) and copy the contents of the local file into the server one). This might be possible with simple text files, but it is nearly impossible with more complex data, like graphics files, databases, etc. etc. This should work (and does NOT):
After these asctions synchronization should understand that the user wants to keep local changes and upload them to the server. However it does not work now. It sais, that the server responded that the file is LOCKED. I use client 2.5.1 on Linux Mint |
|
Am I missing an obvious "bulk resolve" to select local versions of all files currently in conflict? My server was down for a while; now all of applications are loading stale data from the server because my fresher local files have been renamed. |
|
@lowlyocean There's currently no bulk-resolve option. #5896 is an issue about providing more conflict resolution ui help but that's not yet scheduled or even fully thought through. Until then, check out the renaming console commands mentioned above. The client tries hard to avoid creating unnecessary conflict files. For example it'll only ever do it in situations where the server reports a file change (i.e. the server file's etag differs from the locally recorded etag). If you still have information about the situation that caused conflict files to be created in a "pure backup" situation that'd be very much worth a report. Optionally limiting client behavior for "pure backup" operation may be a worthwhile discussion to have. |
|
It concerns me that conflict resolution is proposed as a server-side task. I'm running on a raspberry pi and try to avoid the web interface as much as possible because it is slow to load. The more the client can do, the better. If I'm not using nextCloud for collaboration, I shouldn't see any conflicts. It should be assumed my local copies are the most recent and back them up to the server. |
|
I agree that conflict resolution is usually a local task, that ticket is about giving users better support for resolving conflicts locally through the sync client. |
|
Good work so far, I've noticed lots of improvements like reduced time for client scan for changes. Very much looking forward to DeltaSync feature |
and others
The clients conflict file handling is not optimal currently because we decided to always prefer the server version over the local changes. That makes us moving the local changed file aside with a conflict file name and download the server version in place.
For the user that means that the local changes of the file are never synced to the server. #3222 #4079
Under the assumption that we want to stick with the server version wins policy we could improve this by supporting version branches in the servers versions app as described here: owncloud/core#23002
Once the client detects a conflict file, it uploads the current local version to the servers version app along with the ETag of the file's predecessor, which is known to the client. That way, the local file version would be saved on the server and data loss would be avoided.
This depends on owncloud/core#23002
The user experience would benefit even more from considering the version branches in #4556
@MTRichards @karlitschek @moscicki @jancborchardt @ogoffart @rullzer your opinions please.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The text was updated successfully, but these errors were encountered: