Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File corruption for ecryptfs files #24416

Closed
Redsandro opened this issue May 3, 2016 · 15 comments
Closed

File corruption for ecryptfs files #24416

Redsandro opened this issue May 3, 2016 · 15 comments

Comments

@Redsandro
Copy link

Redsandro commented May 3, 2016

Steps to reproduce

  1. Sync an ecryptfs share with OwnCloud
  2. Use for a couple of months
  3. Some files get corrupted

Expected behaviour

No problems

Actual behaviour

Corrupted files. Trying to chmod recursively shows problems like this. I think less than 1% of my files is affected:

chmod: cannot access ‘./etc/(Klad)/Games Cheats/GTA San Andre\004’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Games Cheats/\236\343\355UQo!<\325’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Computers/\354"|\207\255\351\305a\027TA\037\250\222@\340\343’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Computers/\354\210\177\201\031\t\257GG\037\336\025m\313\317Pu’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Computers/Sand\a\255\003\276Iv;’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Computers/San\255\246\030\003\032\270\023[\266\363\r\250\003=3\343’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Computers/lin0\333Q>\367\376/\035\3254\220\206l\343\302?’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/Computers/The BIOS Optimizatioمr\347\003E\251\355\336X\0200\376\276M\032’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/(Uitzoeken)/m})ME\t\230\272*?U\324@\371\235Nz’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/(Uitzoeken)/Boo\227\f\260\316\361\251\303o\221M\0372 \201\231{’: No such file or directory
chmod: cannot access ‘./etc/(Klad)/(Uitzoeken)/Backtracking EM\337\302lֹ\361\242ῴtA\351OR\273’: No such file or directory

Trying to ls in one of the dirs with problems:

$ ll
ls: cannot access �"|����a�TA���@��: No such file or directory
ls: cannot access �����   �GG���m��Pu: No such file or directory
ls: cannot access Sand���Iv;: No such file or directory
��=3�: No such file or directory
ls: cannot access lin0�Q>��/
�4��l��?: No such file or directory
ls: cannot access The BIOS Optimizatioمr��E���X�0��M�: No such file or directory
total 1.2M
drwxr-s--- 2 sander sander 12K Mar 23 00:43 ./
drwxr-s--- 12 sander sander 4.0K Mar 23 00:46 ../
-rw-r----- 1 sander sander 637 Mar 8 2013 2003-07-01 Comp Specs.txt
-????????? ? ? ? ? ? ?"|????a?TA???@??
-rw-r----- 1 sander sander 6.4K Mar 8 2013 autostart.txt
-rw-r----- 1 sander sander 21K Mar 8 2013 Computer Salland.html
-????????? ? ? ? ? ? ???????GG???m??Pu
-rw-r----- 1 sander sander 2.3K Mar 8 2013 Journalling points.txt
-????????? ? ? ? ? ? lin0?Q>??/??4??l???
-rw-r----- 1 sander sander 748 Mar 8 2013 linux partities.txt
-rw-r----- 1 sander sander 2.5K Mar 8 2013 poorten.txt
-rw-r----- 1 sander sander 359 Mar 8 2013 Reformat protocol.txt
-????????? ? ? ? ? ? San???????[?????=3?
-rw-r----- 1 sander sander 133 Mar 8 2013 Sander HTMLtips.txt
-rw-r----- 1 sander sander 2.0K Mar 8 2013 Sander, lees mij.txt
-rw-r----- 1 sander sander 1.5K Mar 8 2013 Sander RegTweaks.txt
-????????? ? ? ? ? ? Sand????Iv;
-rw-r----- 1 sander sander 564 Mar 8 2013 San Windowstips.txt
-rw-r----- 1 sander sander 858 Mar 8 2013 StopServices.bat
-rw-r----- 1 sander sander 770K Mar 8 2013 The BIOS Optimization Guide v6.2.pdf
-????????? ? ? ? ? ? The BIOS Optimizatioمr??E???X?0??M?
-rw-r----- 1 sander sander 739 Mar 8 2013 Winprogs.txt
-rw-r----- 1 sander sander 12K Mar 8 2013 XP Services optimize.txt

Server configuration

Not sure, using Stack from TransIP - stackstorage.com

Client configuration

Operating system: Ubuntu 14.04 LTS

ownCloud version: 2.1.1
deb http://download.opensuse.org/repositories/isv:/ownCloud:/desktop/Ubuntu_14.04/ /

Logs

ownCloud log (data/owncloud.log)

There is this a hundred times. Not sure if it's relevant.

#=#=#=# Syncrun started 2016-05-03T13:55:50 until 2016-05-03T13:55:50 (614 msec)
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiABbDc7CCY4watlc1pQiAn---/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKlGcTZUvavSidMVinJ6xDIkMtqdYB0WEkrbImE1b7x1Ny9r3JBk.OAMw3yXZLtq90CdFDIK6Qy6mN9WgvvDm8Ck-|INST_NEW|Up|1423086143||114688||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiABbDc7CCY4watlc1pQiAn---/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKlGcTZUvavSidMVinJ6xDIkMtqdYB0WEkrbImE1b7x1Ny9r3JBk.OAMw3yXZLtq90CdFDIK6Qy6mN9WgvvDm8Ck- - server replied: Not Found|0|0|0|||INST_NONE|
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiABbDc7CCY4watlc1pQiAn---/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti67PA.OsdS2y18EAxsE39uttLwXaOVbGZNdVdAshyvwqvMQAyJfd98HqEEn8tlGfTNngvnVWDJo7M4FMOMOc00mFoJck6opP2PN93mGont4A-|INST_NEW|Up|1423086860||24576||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiABbDc7CCY4watlc1pQiAn---/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti67PA.OsdS2y18EAxsE39uttLwXaOVbGZNdVdAshyvwqvMQAyJfd98HqEEn8tlGfTNngvnVWDJo7M4FMOMOc00mFoJck6opP2PN93mGont4A- - server replied: Not Found|0|0|0|||INST_NONE|
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiRVspzaywIFGz1IQC73fDjk--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiwcI.aKhMis3yoEgmPgfikeZJkwsTO-Jbh0WopXkdI7bl4DSfC7t20F1Ct3m6QAU9GlIhx223PnsKo6Zev-IY4YCUCeGa3tsq74YZ6Rf30ME-|INST_NEW|Up|1362752767||24576||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiRVspzaywIFGz1IQC73fDjk--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiwcI.aKhMis3yoEgmPgfikeZJkwsTO-Jbh0WopXkdI7bl4DSfC7t20F1Ct3m6QAU9GlIhx223PnsKo6Zev-IY4YCUCeGa3tsq74YZ6Rf30ME- - server replied: Not Found|0|0|0|||INST_NONE|
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtitTK4qZxjlb54eafB-qxufE--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiKDbUVFaRk9YgUpuLidK3W59MQlCTPnjVn4ANCVE21i6-/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKmxrGY9CgLgx67LEEh45pwWpSDC5mqAKu60IN6QPfWJY3ek5VlJAv8bF4wMBYTTsrR5oNzqVtTwTTK4JnFDkxwU-|INST_NEW|Up|1362517221||446464||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtitTK4qZxjlb54eafB-qxufE--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiKDbUVFaRk9YgUpuLidK3W59MQlCTPnjVn4ANCVE21i6-/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKmxrGY9CgLgx67LEEh45pwWpSDC5mqAKu60IN6QPfWJY3ek5VlJAv8bF4wMBYTTsrR5oNzqVtTwTTK4JnFDkxwU- - server replied: Not Found|0|0|0|||INST_NONE|
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiutnVYiuViNGJTfu4H0dcyU--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKrraBoIHH5PN8nL.5b4MKW--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiFldsaaML7HGSbyabONAQ84N1J.BBosgXgeoHLGLgr964ZgGgQApvnsmGBLIr.ttoZ6NvAOuhkz2Xrjs-W5tFPLBqXAOUfxSsj5rDdhUUBgg-|INST_NEW|Up|1413151080||69632||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiutnVYiuViNGJTfu4H0dcyU--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKrraBoIHH5PN8nL.5b4MKW--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiFldsaaML7HGSbyabONAQ84N1J.BBosgXgeoHLGLgr964ZgGgQApvnsmGBLIr.ttoZ6NvAOuhkz2Xrjs-W5tFPLBqXAOUfxSsj5rDdhUUBgg- - server replied: Not Found|0|0|0|||INST_NONE|
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiutnVYiuViNGJTfu4H0dcyU--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKrraBoIHH5PN8nL.5b4MKW--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiFldsaaML7HGSbyabONAQ84N1J.BBosgXgeoHLGLgr964ZgGgQApvnsmGBLIr.ttoZ6NvAOuhkz2Xrjs-W5tFPQjIqgj18y1RK3tqQb-7NW6-|INST_NEW|Up|1413151098||126976||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiutnVYiuViNGJTfu4H0dcyU--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKrraBoIHH5PN8nL.5b4MKW--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiFldsaaML7HGSbyabONAQ84N1J.BBosgXgeoHLGLgr964ZgGgQApvnsmGBLIr.ttoZ6NvAOuhkz2Xrjs-W5tFPQjIqgj18y1RK3tqQb-7NW6- - server replied: Not Found|0|0|0|||INST_NONE|
|0|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiutnVYiuViNGJTfu4H0dcyU--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKrraBoIHH5PN8nL.5b4MKW--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtibeHYJ.XyrLNw2ufS6cHYjxDmKemd3TF1u2OEN5IxeQUI40BPWoJ2Ummpm-9Dq10aoTRxQ3N-JEqEOUlXvPqIW-k.Y6CKhOYCLfYh3Ud.uKY-|INST_NEW|Up|1413148454||28672||6|The item is not synced because of previous errors: Error downloading https://redsandro.stackstorage.com/remote.php/webdav/Documents.enc/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtiutnVYiuViNGJTfu4H0dcyU--/ECRYPTFS_FNEK_ENCRYPTED.FXbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti-546qsXxOqMXP71D7i9BKrraBoIHH5PN8nL.5b4MKW--/ECRYPTFS_FNEK_ENCRYPTED.FabVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtibeHYJ.XyrLNw2ufS6cHYjxDmKemd3TF1u2OEN5IxeQUI40BPWoJ2Ummpm-9Dq10aoTRxQ3N-JEqEOUlXvPqIW-k.Y6CKhOYCLfYh3Ud.uKY- - server replied: Not Found|0|0|0|||INST_NONE|

All other lines are like this:

20:14:41|429|ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgtipg1byUk6mw8Xskza4GymXE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-KfgtihE8z2vhd0CKln8xkjhZUhE--/ECRYPTFS_FNEK_ENCRYPTED.FWbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti67PA.OsdS2y18EAxsE39uk--/ECRYPTFS_FNEK_ENCRYPTED.FYbVE5oHiOaZfEYE1eWIZf5Wy.-Hfg-Kfgti67PA.OsdS2y18EAxsE39uyWN27y91JA5AjphhkTvN9R-d-a.pUbWN43aTll.c0Bv|INST_SYNC|Down|1431443181|144AD962BB9D5D8D11000|69632|42631|4||0|69632|1431443181|||INST_NONE|

I'm not sure if this is OwnCloud or ecryptfs related, so I posted an ecryptfs related bug here: https://bugs.launchpad.net/ecryptfs/+bug/1577740
However, I did not notice any of this before I switched from Dropbox to OwnCloud. Either that points to OwnCloud, or it's just a coincidence.

@PVince81
Copy link
Contributor

PVince81 commented May 3, 2016

I also use ecryptfs locally, but I sync the mounted part that is non-encrypted. (I also have encryption enabled on the server)

So you are syncing the encrypted ecryptfs files with scrambled file names. I suspect that this will not work correctly because it is likely that ecryptfs will use special characters that cannot be synced by ownCloud properly. Well, actually the sync client should refuse to sync such files.

Maybe you discovered a set of special characters that aren't blocked but get mis-encoded in URLs or so, maybe even normalized by the server.

I'd recommend using ecryptfs with non-scrambled file names instead.

@Redsandro
Copy link
Author

Redsandro commented May 3, 2016

@PVince81

I'd recommend using ecryptfs with non-scrambled file names instead.

Interesting. I'm not sure if it's legal in my country to store personal details (e.g. documents about clients with their name (which is a personal detail) in the filename) unencrypted on a third party computer, so it would require a lot of work to anonymize filenames, and I would lose some convenience. But interesting.

On the other hand, ecryptfs scrambled filenames are [A-z0-9\.\-], which should be fine.

Also, permissions aren't stored in the filename, right? How do entries like this happen?

-????????? ? ? ? ? ? San???????[?????=3?

@PVince81
Copy link
Contributor

PVince81 commented May 3, 2016

@Redsandro actually all I said above is theory, was just guessing.

If ecryptfs uses only ASCII chars for scrambling, then it should work.

Looking at your post above, it looks like even the file names get corrupted ? Or was it the file contents ?

@Redsandro
Copy link
Author

@PVince81 Yes, especially the filenames. I actually cannot verify in all certainty if the contents got corrupted, because I cannot open the files anymore. xyz does not exist when I try to open xyz.

@PVince81
Copy link
Contributor

PVince81 commented May 3, 2016

It almost looks like your FS itself got corrupted somehow.

I don't see any way your file names could get corrupted that way by ownCloud.
It's likely that it's a bug in ecryptfs itself, maybe triggered by the way how the desktop client accesses the FS APIs

@PVince81
Copy link
Contributor

Are you still seeing this issue with OC 9.0.4 ?

@Redsandro
Copy link
Author

9.0.4?

I'm pretty far from that, and it claims to be the latest version.

image

@PVince81
Copy link
Contributor

you're looking at the client version.

9.0.4 is the server version

@Redsandro
Copy link
Author

Oh I wouldn't know. Is there a way to figure this out without having control over the server?

Server configuration
Not sure, using Stack from TransIP - stackstorage.com

@PVince81
Copy link
Contributor

@Redsandro yes, log in on the server with the web UI, then go to the personal page.
If you don't have web UI access for some reason, you can do this:
curl https://host/status.php and it will return the version in an XML response

@Redsandro
Copy link
Author

Redsandro commented Sep 20, 2016

@PVince81

I guess I'm out of luck. The server (out of my control, controlled by stackstorage.com) is:

{"installed":true,"maintenance":false,"version":"8.2.1.4","versionstring":"8.2.1","edition":""}

If you have a commit pointing to a file corruption fix, maybe I can convince them it's important enough to upgrade hundreds of servers.

@PVince81
Copy link
Contributor

8.2 will be EOL in June 2017. That should also be a reason to update.

@PVince81 PVince81 added this to the backlog milestone Jan 27, 2017
@PVince81
Copy link
Contributor

PVince81 commented Apr 6, 2017

Problem might be related to long file names: #25425
Closing as duplicate

@PVince81 PVince81 closed this as completed Apr 6, 2017
@ownclouders
Copy link
Contributor

Hey, this issue has been closed because the label needs info is set and there were no updates for 14 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.

@lock
Copy link

lock bot commented Jul 31, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants