Files getting set to 0 in data folder #3056
Comments
|
Thanks for the detailed report! Is there anything in the nextcloud.log or in your apache/nginx error.log? |
|
Sorry the formatting mess... I'll have to investigate the logs when I'll find some time. The problem is, the only trigger for me to realize there's zeros coming was when opening a file in Collabora, CODE got stuck. Then I saw the file went zero, but didn't have the time to investigate further (besides of restoring the file from backup). The "big" research I did with finding the 215 files was much later, like 2 days later or so... |
|
I also have that issue. Unfortunately i can't say when it happened. Therefore i'm unable to look at the log files. I only can warn everyone about this behavior! Here are some more users of Nextcloud with that zero bytes bug: |
|
For one of the users it could have been the client which overwrites the file: Does this only concern users who sync their files with the nc-client? All longer existing setups with upgrade history? Zero files also happen on local storage, non shared files & folders? If you share a bit more about the other configs, it's easier to find a pattern. |
|
@tflidd - well, for me it's indeed just for the only user syncing with desktop clients (Linux & Windows). All other user's files haven't been touched appearently. Local storage only, non shared files or folders. Quite "simple and small" personal setup. I didn't do the check on the company's NC yet though. @Sanookmakmak - let's try to approach this issue in a calm way. Creating some panic doesn't help. Please give some closer insight on the zeroed files. I know it's boring - that's how it felt when I created the issue yesterday. But maybe this way we can advance faster on it. thanks |
|
I can only say that i use the Linux and Windows client, sorry. I've uploaded the files about half a year ago and i really don't have an idea when and why the files have been zeroed. I found them coincidentally. But i will keep an eye on it if it happens again. And i don't have a backup because Nextcloud is my backup solution. That's why i feel i little bit panic :-( |
|
I see - Be careful, it's "dangerous" and not recommended to use OC or NC as the sole backup solution, even if you would use the "versions" app. The NC server data should always be backupped. That's what the big cloud providers also do. Otherwise they would be in big trouble. Can you provide some information about the system setup? Operating System Thanks |
|
This information is from today, but the issue could also happened half a year ago and of course there were older versions installed. As i told before, i can't say when those files were zeroed. Operating System Kernel (uname -a) Webserver engine (apachectl -V) Database (mysql --version) PHP Version (php --version) |
|
Great. If you find the time, could you analyze your "find $nextcloud_data_folder -size 0 -type f" command in an editor and filter down the major folders of the affected files (more or less as I did in the issue description). And "sudo -u www-data php /var/www/your_nc_folder/occ app:list" - so the devs and ops got even more stuff to work with :) |
|
find $nextcloud_data_folder -size 0 -type f php occ app:list |
|
@icewind1991 @rullzer any idea? Seems like filesystem or syncing causes an issue. |
|
I've posted a script here https://help.nextcloud.com/t/files-become-zero-bytes/7214/17 which will help alert close to the time it happens. The Apache logs don't seem to help much, I caught them when one file was zero'd and they just showed one client putting and the other getting. Anyone know which log file is going to be best to look at and would turning on debugging help? |
|
Can you post those two lines? The GET and the PUT? |
|
Sure, The last good backup of the file in question was on the evening of the Jan 1st, by the evening of the 2nd it was zero bytes. So entries from web logs from the 1st and 2nd of Jan should cover it, I have the following: (49 is my desktop, 34 is my laptop) 192.168.0.34 - me [02/Jan/2017:10:18:02 +0000] "GET /remote.php/webdav/Documents/Passwords/passwords.kdb HTTP/1.1" 200 123515 "-" "Mozilla/5.0 (Linux) mirall/2.2.4" And here's what looks relevant from the nextcloud.log {"reqId":"Lkxh6CnAZ4QR+Y6jP+eo","remoteAddr":"192.168.0.49","app":"admin_audit","message":"Login attempt: "me"","level":1, |
|
Without any further info I'm inclined to believe that this is a client side issue, either a bug in the sync client or some situation on the client seed causing the sync client to see a 0 byte file. To further investigate the situation you can try applying this patch which will log the stacktrace anytime a 0 byte file is written |
|
No feedback since half a year. I will close this. Once you have more details we can of course reopen this ticket. |
|
I had the same problem in 2012 and in 2015 (inbetween and after that I stopped using Owncloud/Nextcloud for file sync). I found several issues like this. I wanted to link to the issue below, that seems to have some more details according to this phenomenon. The first report of this is from 2013: I have hosted Owncloud on all-inkl.com, so I can't say much about the server setup. I used the owncloud client on windows for synchronization. I found several pdf-files synced down to 0 bytes. On my computers as well as on the server. Furthermore, I could not restore old versions of these files. |
|
hello, experienced something similar today on NC 13.0.0 ( originally 12.0.5 updated to 13.0.0 without errors ) Nextcloud server 13.0.0 (12.0.5 updated) Created a few days ago some folder, and copy/past files/pic to it and from nextcloud.log file: |
|
This also happens to me on a regular basis. The files seems to get 0ed on the server storage, not on the client side. A Please reopen this issue as it is really frightening to get files erased randomly. I'm available to do some tests if needed. For what it's worth it happened on several files when moved from a subfolder to another, in the same synced folder. I use a Windows client and several Linux clients (stable repository). Client version (Windows): 2.3.3. Server version: 12.0.4, on Debian Stretch. |
|
Did you try to add the extended logging from #3056 (comment)? |
|
I fortunately didn't have these problems anymore, after my last clean-up (about an year ago). It think it was a combination of Linux/Windows desktop clients and an old Linux owncloud client, but I'm not 100% sure of it. Sorry - I can't remember and am not able to describe the solving. It was one of these moments of "oh ok, it didn't happen anymore". |
|
@tflidd NC12 does not fix the issue, for 6 shared folders I have empty files appearing in 3 tonight, several days after the upgrade. I will try to enable extended logging tomorrow. |
|
This patch seems to cause an issue: {"reqId":"peDOLrsekBLqkxn50o5W","level":3,"time":"2018-04-05T19:30:23+00:00","remoteAddr":"192.168.1.1","user":"<edited>","app":"PHP","method":"GET","url":"\/remote.php\/dav\/files\/<edit>.jpeg","message":"Undefined variable: stream at \/var\/www\/nuaj\/lib\/private\/Files\/View.php#1003","userAgent":"Mozilla\/5.0 (Linux) mirall\/2.3.3 (Nextcloud)","version":"13.0.1.1"} |
|
@icewind1991 it was your patch, can you take a look? |
|
The topic on the forum is still active: I looked through my setup as well and found a lot of 0-bytes files. I have to hunt them down and check in older backups, if I can find the actual files again and perhaps find why this happened. Reopening. |
|
@Subito I'm not using Nextcloud on iOS but on Android. Some of the 0 byte files were pictures. But most of the files were never written to NC from a mobile device. Like LibreOffice files. Those files were mostly synced by the Linux/Windows desktop client. Since the issue seems to affect the WebDAV interface, I guess this should be reproduceable on all desktop and mobile clients. In my config So I tried to put an Ubuntu 20.10 Server iso image (998MB) and a zip archive with 206MB in my Nextcloud sync folder on Windows 10. This worked well, they have their expected file size in the nextcloud data dir of the account. I'll try to do the same with Linux. To make sure that those clients are really affected, I think we need to capture the requests between the sync client and the server to see if the |
|
I noticed a new 0 byte file: As a result, it seems that the Android app is (also) responsible for destroying files with 0 byte content. |
|
We added this in the Nextcloud VM to easily check for 0-byte files: |
|
Been having the same issue - when uploading a file through webdav all files were being zero-ed. Having just checked and uploading through the web interface does not cause the problem. Interesting! So it seems my client is causing this issue? |
Which webdav client were you using? |
I was using MountainDuck (CyberDuck). |
So seems like MountainDuck (CyberDuck) isn't compatible with Nextcloud then? |
After going through troubleshooting it seems its to do with the configuration on the server. https://trac.cyberduck.io/wiki/help/en/howto/mount/issues/fastcgi "Using a client to upload files with HTTP chunked transfer encoding to a server with fastcgi/php-fpm enabled can lead to zero-byte files" |
Yeah, I've seen that before. To be able to use MountanDuck you need to change to Apache PHP. I'm quite surprised the MountainDuck devs haven't worked around this issue yet. |
Since april, I just had zero byte files on one account. The affected files were uploaded using the NC Android app with the auto upload function. At least my own NC account contains a lot of files in different sizes. A part is also encrypted with Cryptomator. However, there are no 0 byte files any more. The reason is unclear. You can work around this by setting |
To be clear, that's an Nginx settting. |
|
We still struggle with zero byte files using mountain duck. A couple a week across my client base (primarily with pdf files). We don’t have the problem with native windows WebDAV client. We are using an Apache reverse proxy to a docker Apache mod_PHP Nextcloud. We have put tickets in with mountain duck but after reviewing the logs they have no answers.
…________________________________
From: Daniel Hansson ***@***.***>
Sent: Friday, July 23, 2021 3:31 AM
To: nextcloud/server
Cc: bbolokofsky; Comment
Subject: Re: [nextcloud/server] Files getting set to 0 in data folder (#3056)
fastcgi_request_buffering on
To be clear, that's an Nginx settting.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#3056 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AH5OEDMYQ24FNL4FBGN3ZQTTZFADNANCNFSM4C4JMZJQ>.
|
|
I was affected by this issue, too, and after too many hours of searching for a solution, I found this issue report: So I did exactly that (by using Ondřej Surý's PPA) and bingo, no more zero byte files when using Mountain Duck to copy files to Nextcloud.
|
|
I have the same issue, when I upload PDF files via the Webbrowser (Firefox on Mac). Even PDFs I uploaded minutes ago, I cannot open any more. When I open the file I get this error. But the file in the files folder shows me still the correct size: NC version: 22.2.3 Edit: It seems, that only files I recently uploaded are affected. File, which are older than 23 days, are not affected at all. So it seems, this "bug" was introduced with the Version 22.2.2 (or 22.2.1). |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
Hi, please update to at least 23.0.12 and report back if it fixes the issue. Thank you! |
szaimen
added
needs info
0. Needs triage
|
@Zealot2000 thanx for solution, but in my case this wasnt enought. second part of solution - fixing nginx reverse proxy (in my case) by adding following lines to nginx config: as described here |
and others
Steps to reproduce
Expected behaviour
Files should not randomly getting set to 0
Actual behaviour
Hi guys,
I'm one of those losing files to zeros. I thought it's better to continue in here, rather than spamming the forum's thread about this issue.
I'm on NC11RC1 since yesterday and I do have 215 files in the data folder set to zero - but I for sure already had one file on NC11 final that was set to zero and I had to restore it from a backup. Maybe the other 214 were lost too already on NC11 final.
a) 13 files related to one (and only one) frequent user's data
10 files in ncdata/ncuser01/files/01_Daten/
File extensions: .ini .txt .docx .exe .cfg .bin
3 files:
ncdata/ncuser01/files_versions/MyCloud/Tesfile01.txt.v1474618386
ncdata/ncuser01/files_trashbin/files/New Text Document.txt.d1475085009
ncdata/ncuser01/files_trashbin/files/New Text Document.txt.d1480781891
b) 199 files related to "updater-data" in the data folder
187 files in ncdata/updater-50793ab339aa5/
6 files in ncdata/updater-data/checkpoint/9.0.0.19-570dfac02c766/apps/
6 files in ncdata/updater-data/checkpoint/9.0.1.3-573246e809595/apps/
c) 3 other files in data root:
ncdata/.ocdata
ncdata/index.html
ncdata/appdata_50793ab339aa5/preview/16024/64-64-crop.png
It's midnight now, and I came home from work at 10pm, so please don't ask for more infos now.
Thanks, kind regards
Server configuration
Debian Jessie 8.6 (x64) - SMP Debian 4.8.11-1~bpo8+1 (2016-12-14) x86_64 GNU/Linux
Apache/2.4.25 (Debian)
mysql Ver 15.1 Distrib 10.0.28-MariaDB
PHP 5.6.29-0+deb8u1
NC 11.0.1 RC1 (beta)
Updated from NC11 final with Webgui updater
**Original clean manual installation of Nextcloud 9 (from nextcloud.com download) and then additionally migrated data from an old owncloud installation (which itself came the long way from OC 3.X) **
Signing status:
Signing status
Integrity checker has been disabled. Integrity cannot be verified.`List of activated apps:
App list
Enabled:
Disabled:
The content of config/config.php:
Config report
Are you using external storage, if yes which one: No
Are you using encryption: no
Are you using an external user-backend, if yes which one: ActiveDirectory (but not really used, still local users)
LDAP config
Errorlog from when?
Nextcloud log from when?
From when?```
The text was updated successfully, but these errors were encountered: