deleted files locally (but still on server) after 2.4 update - Ubuntu

[simogeo]

Expected behaviour

Synchronised files / folders between my PC and the cloud server

Actual behaviour

30 Gb of files just disappeared locally on my computer. After 2.4 client update I guess.
It seems the issue is similar to #3102.

Is there a way to downgrade owncloud client version on ubuntu ? I would be (very) happy to get back my files.

Steps to reproduce

1. Use Ubuntu
2. update to 2.4 client version
3. switch of the computer, go to bed, boot early morning, see ... there is no files or just few !?

Server configuration

Operating system:

Web server: Apache 2

Database: MySQL 5

PHP version: 5.6

ownCloud version: 12.04

Storage backend (external storage):

Client configuration

Client version: Version 2.4.0 (build 8911)

Operating system: Ubuntu xenial

OS language: fr

Qt version used by client package (Linux only, see also Settings dialog):
Qt 5.6.2

Client package (From ownCloud or distro) (Linux only): ownCloud

Installation path of client:

Logs

extract of server logs :

78.194.240.218 - simon [03/Jan/2018:19:02:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:02:35 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1601 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 90.12.217.113 - laurent [03/Jan/2018:19:02:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:02:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:03:21 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:03:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:03:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:03:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:04:21 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:04:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:04:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:04:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 78.194.240.218 - simon [03/Jan/2018:19:05:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 2863 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:05:23 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 2865 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 90.12.217.113 - laurent [03/Jan/2018:19:05:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:05:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:06:21 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:06:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 78.194.240.218 - simon [03/Jan/2018:19:06:37 +0100] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 884 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:06:51 +0100] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 884 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 90.12.217.113 - laurent [03/Jan/2018:19:06:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:06:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:07:21 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:07:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:07:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:07:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:08:21 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:08:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:08:51 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:08:52 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)" 90.12.217.113 - laurent [03/Jan/2018:19:09:21 +0100] "PROPFIND /remote.php/dav/files/laurent/ HTTP/1.1" 207 1600 "-" "Mozilla/5.0 (Windows) mirall/2.3.2 (build 6928)" 78.194.240.218 - simon [03/Jan/2018:19:09:22 +0100] "PROPFIND /remote.php/dav/files/simon/ HTTP/1.1" 207 1598 "-" "Mozilla/5.0 (Linux) mirall/2.4.0 (build 8911)"

[simogeo]

even stranger,
My client starts to sync again but some files only ... Most of them are not downloaded locally ....

[ogoffart]

Hi,
Your problem sounds serious. But I doubt it is related to #3102 in any way.

I would need more information to be able to debug that problem. Like the full log of a sync, that show the discovery and the reconcile phase.
I need to understand why the files were suddenly gone.

[simogeo]

hi @ogoffart : thanks for your support ! Here we go for full logs
https://gist.github.com/simogeo/69a57034acb6db4957a6b5be1c3eb68f

in the meantime, is there a way to downgrade client software ? I would appreciate to be able to work on good conditions again

[ckamm]

@simogeo For downgrading, I presume you can use the isv:/ownCloud:/desktop:/client-2.3.4/Ubuntu_16.04/ repository instead of isv:/ownCloud:/desktop/Ubuntu_16.04/ to grab the 2.3.4 snapshot. Note I haven't tried that.

Could you give us (if necessary by email; mail at ckamm de) a client log file? You can generate it by starting the owncloud client with --logfile mylog.txt. That will allow us to see what your client is thinking about the various missing files. Also let us know about the name of a file that used to be on your computer but is no longer there.

[ogoffart]

Thanks for that (server) log.
On the webdav endpoint, It shows lots of PROPFIND which all seems successfull (http code 207) and a couple of PUT and MOVE.

Do you have the client log of a sync ? (from --logfile)

Thanks for helping us investigate this issue.

In order to downgrade, you can just run the installer of the previous version [ https://download.owncloud.com/desktop/stable/ ] Downgrading should somehow work altough it is not really supported (new created accounts or folder might not be imported). And in particular, i doubt this will re-download all your files.

Also, what exact version did you upgrade from?

One theoryof what might be causing the bug is that the upgrade caused a full remote discovery (We do that when upgrading from 2.2, but not from 2.3. However, maybe the remotePerm storage change might have changed the order of permission and we would still do a full discovery in that case)
The server might have missed some entries in the PROPFIND replies and the client would then delete them.
You can re-force a full discovery by pressing CTRL+F6, or adding pattern in the ignore list.
Does the file re-appear then?
Please save the log (with --logfile) when doing that.

I'd appreciate if you waited a bit before attempting to downgrade because it would be usefull to know what caused this issue.

[simogeo]

@ckamm : Here is the right url it seems : http://download.opensuse.org/repositories/isv:/ownCloud:/desktop:/client-2.3.4/Ubuntu_16.04/. Not tested yet, since I will try to help you to debug this.

Not that when I stop and start again owncloud client with logfile option, it deleted some files ....
I sent you a link to download log

@ogoffart : If you send me your email, I'll forward you the link to check logs

[ckamm]

@simogeo Thanks for the log, will investigate from there!

[simogeo]

I 've tried Ctrl-F6 ... Owncloud started to sync again but it seems, it did not restore all files / folders.
ALl this, is kind of scary ...

@ogoffart, @ckamm : I can give you access to the huge (more than 300MB) log file if needed.

I think I will try to downgrade before monday because the situation is really getting paralytic .... let me know if I can help before

[simogeo]

Here is where I am (not a really good news ...) :

• I've downgraded my client to 2.3.3 from https://download.owncloud.com/repositories/desktop/download/ (see Can't install on Ubuntu 16.04 (latest) #6034 and can't install owncloud client 2.4.0 on ubuntu 16.04 #6267).
• I started the sync again.
• I was able to appreciate a new synchronisation and to believe everything want right. Let the computer work to sync my data.
• I came back again and observed the same trouble. All was downloaded and then most of the files were deleted again .. :-(

Here is the owncloudsync.log file found in the local root folder. where I can see lots of INST_REMOVE instances. (I don't know nothing of owncloud, so I don't know if it helps or not).

What I am trying now :
I removed the account, and created a new one downloading everything again to a new local folder.

I will let you know what will happen .... Thanks again for support.

[ogoffart]

@simogeo thanks for your investigations.

Wodering if this is a regression in 2.4 or a problem that existed before, also in 2.3.
The owncloudsync.log shows that the files are indeed deleted, but it does not tell why. Normaly this happens when the server does not report these files anymore, so this could be a server problem. I haven't had time to look at the log from --logfile yet, but it should tell us if there is anything fishy with the filesthat got removed, or if the client simply don't "see" them as the server don't report them.

If you create a new account or a new folder sync connection, and that the files does not get downloaded, that would shows that there is most likely something wrong with the server.
But if there is a difference between 2.3 and 2.4, that would be something odd that we need to investigate.

(I was thinking maybe it is an encoding problem? the the log shows that there are also file in pure ascii that are also removed, so probably not)

[simogeo]

I haven't had time to look at the log from --logfile yet, but it should tell us if there is anything fishy with the filesthat got removed, or if the client simply don't "see" them as the server don't report them.

I guess is able to see them because it download them, and then only deleted them ...

Creating a new account and downloading everything again seems to work. I'm still using 2.3.3 client version and won't test upgrading again right now.

Still, I'd be happy to understand and to be able to upgrade again my client preventing a big mess. So please, let me know if you find something.

Thanks all for your support,