New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete file fails #26837

Closed
JochenHWezel opened this Issue Dec 18, 2016 · 30 comments

Comments

Projects
None yet
6 participants
@JochenHWezel
Copy link

JochenHWezel commented Dec 18, 2016

I delete a file which only exists on owncloud web front (file itself seems to be already deleted on file server for some reason). The webfrontend displays an updated file listing without that file. Going back to parent directory and going back to the first directory, the file is in the directory listing again!

A failure in log file has NOT been logged. Only after switching log to everything, I get this entry:

Exception: {"Message":"HTTP\/1.1 404 File with name BEI Dienst Predigten\/Predigt-Entw\u00fcrfe Jochen\/Dropbox - Transfer.lnk could not be located","Exception":"Sabre\\DAV\\Exception\\NotFound","Code":0,"Trace":"#0 \/var\/www\/owncloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Tree.php(178): OCA\\DAV\\Connector\\Sabre\\ObjectTree->getNodeForPath('BEI Dienst Pred...')\n#1 \/var\/www\/owncloud\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php(285): Sabre\\DAV\\Tree->delete('BEI Dienst Pred...')\n#2 [internal function]: Sabre\\DAV\\CorePlugin->httpDelete(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))\n#3 \/var\/www\/owncloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\n#4 \/var\/www\/owncloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(459): Sabre\\Event\\EventEmitter->emit('method:DELETE', Array)\n#5 \/var\/www\/owncloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(248): Sabre\\DAV\\Server->invokeMethod(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))\n#6 \/var\/www\/owncloud\/apps\/dav\/appinfo\/v1\/webdav.php(56): Sabre\\DAV\\Server->exec()\n#7 \/var\/www\/owncloud\/remote.php(164): require_once('\/var\/www\/ownclo...')\n#8 {main}","File":"\/var\/www\/owncloud\/apps\/dav\/lib\/Connector\/Sabre\/ObjectTree.php","Line":170,"User":"jw"}

Expected behaviour:

  1. Web UI: delete errors in system lead to an error message confirmation box in UI directory listing that the file wasn't deleted and some error details
  2. A file pointer which points to a non-existing file at the owncloud server's file system should be deleted even if the filesystem is missing the file
  3. The above error message of the debug log should be fixed
@ghost

This comment has been minimized.

Copy link

ghost commented Dec 18, 2016

@JochenHWezel Please fill out the issue template shown when creating a new issue. Currently even the basic info about your environment is missing.

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Dec 19, 2016

Hello,
I have the same problem with oc9.1.1, a folder supposed to be deleted still appears on web interface but no longer exists on server.
[edit] This sub-folder is part of shared folder from another user. [/edit]
I upgraded to 9.1.3 but Web UI still makes this folder visible and report this log message :

Steps to reproduce

Unfortunately, I don't know how to reproduce, it appends.

Server configuration

Operating system:
Debian Wheezy up to date

Web server:
Apache 2.2.22-13+deb7u7

Database:
MySQL 5.5.46-0+deb7u1

PHP version:
PHP 5.6.29-1~dotdeb+7.1

ownCloud version:
9.1.1 when error occurs, 9.1.3 now

Updated from an older ownCloud or fresh install:
Fresh from 9.1.1

Where did you install ownCloud from:
From owncloud server archive file (zip)

Signing status (ownCloud 9.0 and above):

No errors have been found.

List of activated apps:

# sudo -u www-data php occ app:list
Enabled:
  - activity: 2.3.2
  - calendar: 1.4.1
  - comments: 0.3.0
  - dav: 0.2.7
  - federatedfilesharing: 0.3.0
  - federation: 0.1.0
  - files: 1.5.1
  - files_pdfviewer: 0.8.1
  - files_sharing: 0.10.0
  - files_texteditor: 2.1
  - files_trashbin: 0.9.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - files_w2g: true
  - firstrunwizard: 1.1
  - gallery: 15.0.0
  - logreader: true
  - notifications: 0.3.0
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.1
  - updatenotification: 0.2.1
Disabled:
  - encryption
  - external
  - files_antivirus
  - files_external
  - user_external
  - user_ldap

The content of config/config.php:

# sudo -u www-data php occ config:list system
{
    "system": {
        "instanceid": "ocvhfoo34wv1",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.customer.fr",
            "customer.owncloud.ovh"
        ],
        "datadirectory": "\/home\/cloud.customer.fr\/donnees",
        "overwrite.cli.url": "https:\/\/cloud.customer.fr",
        "dbtype": "mysql",
        "version": "9.1.3.1",
        "dbname": "cloudcustomer",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_from_address": "ne-pas-repondre",
        "mail_smtpmode": "php",
        "mail_domain": "cloud.customer.fr",
        "maintenance": false,
        "logfile": "\/home\/logs\/owncloud\/cloud.customer.fr.log",
        "loglevel": 2,
        "log_rotate_size": "20971520",
        "logtimezone": "Europe\/Berlin",
        "default_language": "fr",
        "minimum.supported.desktop.version": "1.7.0",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "forcessl": true,
        "appstoreenabled": true,
        "updatechecker": true
    }
}

Are you using external storage, if yes which one:
No

Are you using encryption:
No

Are you using an external user-backend, if yes which one:
No

Logs

ownCloud log (data/owncloud.log)

{"reqId":"OWtUMgW5kEUMSAcbHjGl","remoteAddr":"109.190.199.43","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 404 File with name Commercial\\\/Plaquettes\\\/Fiches Techniques Trackers could not be located\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\NotFound\",\"Code\":0,\"Trace\":\"#0 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(903): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->getNodeForPath('Commercial\\\/Plaq...')\\n#1 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(334): Sabre\\\\DAV\\\\Server->getPropertiesForPath('Commercial\\\/Plaq...', Array, 1)\\n#2 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPropFind(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#3 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#4 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:PROPFIND', Array)\\n#5 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#6 \\\/home\\\/cloud.customer.fr\\\/www\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(56): Sabre\\\\DAV\\\\Server->exec()\\n#7 \\\/home\\\/cloud.customer.fr\\\/www\\\/remote.php(164): require_once('\\\/home\\\/cloud.okw...')\\n#8 {main}\",\"File\":\"\\\/home\\\/cloud.customer.fr\\\/www\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/ObjectTree.php\",\"Line\":170,\"User\":\"espritlibre\"}","level":0,"time":"2016-12-19T08:06:57+01:00","method":"PROPFIND","url":"\/remote.php\/webdav\/Commercial\/Plaquettes\/Fiches%20Techniques%20Trackers","user":"espritlibre"}
@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Dec 19, 2016

Running occ files:scan --all should help resolve the inconsistency.
Also in 9.1 it should be possible to delete such "ghost" files already in the web UI.

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Dec 19, 2016

I already tried running php console.php files:scan --path="/User/files/Commercial/Plaquettes without success, folders still remains visible on Web UI.
Also, scanning all files will take very long time...
What is the difference between occ command and using console.php?
Thanks for you help

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Dec 19, 2016

Ok, tested with the exact command occ files:scan --all and folders does not disappear on Web UI.
Same error in logs when cliking on it.

@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Dec 19, 2016

Hmm, strange... scanning usually resolves that problem.
Let's reformat the stack trace:

{"reqId":"OWtUMgW5kEUMSAcbHjGl","remoteAddr":"109.190.199.43","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 404 File with name Commercial\\\/Plaquettes\\\/Fiches Techniques Trackers could not be located\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\NotFound\",\"Code\":0,\"Trace\":\"#0 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(903): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->getNodeForPath('Commercial\\\/Plaq...')\\n
#1 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(334): Sabre\\\\DAV\\\\Server->getPropertiesForPath('Commercial\\\/Plaq...', Array, 1)\\n
#2 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPropFind(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n
#3 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n
#4 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:PROPFIND', Array)\\n
#5 \\\/home\\\/cloud.customer.fr\\\/www\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n
#6 \\\/home\\\/cloud.customer.fr\\\/www\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(56): Sabre\\\\DAV\\\\Server->exec()\\n
#7 \\\/home\\\/cloud.customer.fr\\\/www\\\/remote.php(164): require_once('\\\/home\\\/cloud.okw...')\\n
#8 {main}\",\"File\":\"\\\/home\\\/cloud.customer.fr\\\/www\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/ObjectTree.php\",\"Line\":170,\"User\":\"espritlibre\"}","level":0,"time":"2016-12-19T08:06:57+01:00","method":"PROPFIND","url":"\/remote.php\/webdav\/Commercial\/Plaquettes\/Fiches%20Techniques%20Trackers","user":"espritlibre"}
@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Dec 19, 2016

@esprit-libre when deleting from the web UI, you should see a "DELETE" request in the network console of the browser and also a matching log entry. Can you see if you can find that ?

The log entry above is from a separate PROPFIND request, not from a failing deletion.
Would be good to see what the DELETE call says.

@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Dec 19, 2016

btw, I see you're using a memcache. You might want to try clearing that in case there are remains.

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Dec 19, 2016

I use OwnCloud v9.1.3 (stable) on Ubuntu 16.04.1 LTS with most current updates.

root@beiweb01:/var/www/owncloud# sudo -u www-data php occ app:list
Enabled:
  - activity: 2.3.2
  - comments: 0.3.0
  - dav: 0.2.7
  - federatedfilesharing: 0.3.0
  - federation: 0.1.0
  - files: 1.5.1
  - files_external: 0.6.0
  - files_pdfviewer: 0.8.1
  - files_sharing: 0.10.0
  - files_texteditor: 2.1
  - files_trashbin: 0.9.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 15.0.0
  - notifications: 0.3.0
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.1
  - updatenotification: 0.2.1
  - user_ldap: 0.9.0
Disabled:
  - calendar
  - documents
  - encryption
  - external
  - files_antivirus
  - user_external

The file is located in a folder structure shared by another user to me.

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Dec 19, 2016

Also for me, this suggestion didn't work:

occ files:scan --all

This is the owncloud.log entry created only on debug-everything:

{"reqId":"W1U0DDvK8gENlGrBJFEG","remoteAddr":"192.168.32.104","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 404 File with name BEI Dienst Predigten\\\/Predigt-Entw\\u00fcrfe Jochen\\\/Dropbox - Transfer.lnk could not be located\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\NotFound\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(178): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\ObjectTree->getNodeForPath('BEI Dienst Pred...')\\n#1 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(285): Sabre\\\\DAV\\\\Tree->delete('BEI Dienst Pred...')\\n#2 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpDelete(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#3 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#4 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:DELETE', Array)\\n#5 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#6 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(56): Sabre\\\\DAV\\\\Server->exec()\\n#7 \\\/var\\\/www\\\/owncloud\\\/remote.php(164): require_once('\\\/var\\\/www\\\/ownclo...')\\n#8 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/ObjectTree.php\",\"Line\":170,\"User\":\"jw\"}","level":0,"time":"2016-12-19T18:10:22+00:00","method":"DELETE","url":"\/owncloud\/remote.php\/webdav\/BEI%20Dienst%20Predigten\/Predigt-Entw%C3%BCrfe%20Jochen\/Dropbox%20-%20Transfer.lnk","user":"jw"}

This is what the browser presents:
image

image

These are additional logs which might be helpful:
image

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Dec 19, 2016

Here some additional owncloud logs which might help:

{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cli","message":"Memcache \\OC\\Memcache\\APCu not available for local cache","level":1,"time":"2016-12-19T18:15:01+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cli","message":"Memcache \\OC\\Memcache\\APCu not available for distributed cache","level":1,"time":"2016-12-19T18:15:01+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\DAV\\CardDAV\\SyncJob job with ID 1","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\DAV\\CardDAV\\SyncJob job with ID 1","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Federation\\SyncJob job with ID 2","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Federation\\SyncJob job with ID 2","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files\\BackgroundJob\\DeleteOrphanedItems job with ID 6","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files\\BackgroundJob\\DeleteOrphanedItems job with ID 6","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files\\BackgroundJob\\CleanupFileLocks job with ID 7","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files\\BackgroundJob\\CleanupFileLocks job with ID 7","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files_Versions\\BackgroundJob\\ExpireVersions job with ID 8","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files_Versions\\BackgroundJob\\ExpireVersions job with ID 8","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Activity\\BackgroundJob\\EmailNotification job with ID 9","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Activity\\BackgroundJob\\EmailNotification job with ID 9","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Activity\\BackgroundJob\\ExpireActivities job with ID 10","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Activity\\BackgroundJob\\ExpireActivities job with ID 10","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files_Trashbin\\BackgroundJob\\ExpireTrash job with ID 11","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files_Trashbin\\BackgroundJob\\ExpireTrash job with ID 11","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OC\\BackgroundJob\\Legacy\\RegularJob job with ID 14","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OC\\BackgroundJob\\Legacy\\RegularJob job with ID 14","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OC\\Authentication\\Token\\DefaultTokenCleanupJob job with ID 16","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"no app in context","message":"Invalidating tokens older than 2016-12-18T18:15:02+00:00","level":1,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OC\\Authentication\\Token\\DefaultTokenCleanupJob job with ID 16","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\UpdateNotification\\Notification\\BackgroundJob job with ID 17","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\UpdateNotification\\Notification\\BackgroundJob job with ID 17","level":0,"time":"2016-12-19T18:15:02+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\User_LDAP\\Jobs\\UpdateGroups job with ID 18","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\User_LDAP\\Jobs\\UpdateGroups job with ID 18","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\User_LDAP\\Jobs\\CleanUp job with ID 19","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\User_LDAP\\Jobs\\CleanUp job with ID 19","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files_Sharing\\DeleteOrphanedSharesJob job with ID 22","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files_Sharing\\DeleteOrphanedSharesJob job with ID 22","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files\\BackgroundJob\\ScanFiles job with ID 5","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"OC\\Files\\Cache\\Scanner","message":"!!! Path '' is not accessible or present !!!","level":0,"time":"2016-12-19T18:15:03+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files\\BackgroundJob\\ScanFiles job with ID 5","level":0,"time":"2016-12-19T18:15:04+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Run OCA\\Files_Sharing\\ExpireSharesJob job with ID 23","level":0,"time":"2016-12-19T18:15:04+00:00","method":"--","url":"--","user":"--"}
{"reqId":"th++GXOkJwYfJGfmI077","remoteAddr":"","app":"cron","message":"Finished OCA\\Files_Sharing\\ExpireSharesJob job with ID 23","level":0,"time":"2016-12-19T18:15:04+00:00","method":"--","url":"--","user":"--"}
@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Dec 19, 2016

Ok, thanks for the details.

Let's check the database: select * from oc_filecache where path like "%Dropbox - Transfer%".
I expect that this entry still exist in the database but there is no such matching file inside the "data" folder.

Now another question is whether only this file is gone or all the parent folders are gone too inside the "data" folder ?

Locally I've tested on 9.1.3 the following steps:

  1. Create a file "test.txt" in the web UI
  2. Go to the data folder and directly delete "test.txt"
  3. Refresh the web UI: see the file still appears
  4. Delete "test.txt" => no error, file is gone.

So I suspect there is something else at play.

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Dec 20, 2016

Hello,
here is the SQL request:
mysql> select fileid, path, permissions, parent, mimetype from oc_filecache where path like "%Techniques Trackers%"; +--------+-------------------------------------------------------------------------------------------+-------------+--------+----------+ | fileid | path | permissions | parent | mimetype | +--------+-------------------------------------------------------------------------------------------+-------------+--------+----------+ | 1856 | files/Partage/Plaquettes/Fiches Techniques Trackers | 31 | 1829 | 2 | | 1964 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 110m² 260Wc - v1.pdf | 27 | 1856 | 4 | | 1965 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 110m² 260Wc - v1.jpg | 27 | 1856 | 6 | | 1966 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 110m² 285Wc - v1.jpg | 27 | 1856 | 6 | | 1967 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 110m² 285Wc - v1.pdf | 27 | 1856 | 4 | | 1968 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 1er page.pdf | 27 | 1856 | 4 | | 1969 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 20m² - v1.jpg | 27 | 1856 | 6 | | 1970 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 1er page.jpg | 27 | 1856 | 6 | | 1971 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 20m² - v1.pdf | 27 | 1856 | 4 | | 1972 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 40m² - v1.jpg | 27 | 1856 | 6 | | 1973 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 40m² - v1.pdf | 27 | 1856 | 4 | | 1974 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 60m² 260Wc - v1.pdf | 27 | 1856 | 4 | | 1975 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 60m² 260Wc - v1.jpg | 27 | 1856 | 6 | | 1976 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 60m² 285Wc - v1.pdf | 27 | 1856 | 4 | | 1977 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 60m² 285Wc - v1.jpg | 27 | 1856 | 6 | | 1978 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 80m² 260Wc - v1.jpg | 27 | 1856 | 6 | | 1979 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 80m² 260Wc - v1.pdf | 27 | 1856 | 4 | | 1980 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 80m² 285Wc - v1.jpg | 27 | 1856 | 6 | | 1981 | files/Partage/Plaquettes/Fiches Techniques Trackers/fiche technique 80m² 285Wc - v1.pdf | 27 | 1856 | 4 | +--------+-------------------------------------------------------------------------------------------+-------------+--------+----------+ 19 rows in set (0.01 sec)

As you can read folders and files are still present in database.
I received some additional information from the user concerned:

I tried to reproduce without success.

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Dec 20, 2016

This is the result of MySQL:

mysql> mysql> select * from oc_filecache where path like "%Dropbox - Transfer%";
+--------+---------+---------------------------------------------------------------------------------------------+----------------------------------+--------+------------------------------------+----------+----------+------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
| fileid | storage | path                                                                                        | path_hash                        | parent | name                               | mimetype | mimepart | size | mtime      | storage_mtime | encrypted | unencrypted_size | etag                             | permissions | checksum |
+--------+---------+---------------------------------------------------------------------------------------------+----------------------------------+--------+------------------------------------+----------+----------+------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
|  24984 |       1 | files/BEI Allgemein/Gottesdienste/Predigten/Predigt-Entwürfe Jochen/Dropbox - Transfer.lnk  | cd6a0c2c7a609f4a1186f0e7a34342bd |   9180 | Dropbox - Transfer.lnk             |        7 |        5 | 1137 | 1472895342 |    1472895342 |         0 |                0 | c2cb7db6f8dc6e8ea7da9aac5b1bd59a |          27 |          |
|  27366 |       1 | files_trashbin/files/Dropbox - Transfer.lnk.d1476520535                                     | e07c3b52b9fe0e191c3734c2c6c08b62 |   5290 | Dropbox - Transfer.lnk.d1476520535 |        7 |        5 | 1137 | 1474189752 |    1474189752 |         0 |                0 | 815e47d6e356942923c931e6af8bff48 |          27 |          |
|  27369 |       3 | files_trashbin/files/Dropbox - Transfer.lnk.d1476520535                                     | e07c3b52b9fe0e191c3734c2c6c08b62 |  27222 | Dropbox - Transfer.lnk.d1476520535 |        7 |        5 | 1137 | 1474189752 |    1474189752 |         0 |                0 | 7d4550b8254adb0acc9d3b3e74366472 |          27 |          |
+--------+---------+---------------------------------------------------------------------------------------------+----------------------------------+--------+------------------------------------+----------+----------+------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
3 rows in set (0,07 sec)

Now another question is whether only this file is gone or all the parent folders are gone too inside the "data" folder ?

There are about 2 files in the folder with this problem. Other additional files in the same folder are still available.

@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Dec 21, 2016

The workaround would be to manually delete these obsoleted entries.
But I don't understand why occ files:scan doesn't do it automatically on your environment.

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Dec 21, 2016

Ok. Strange.
Only oc_filecache table is concerned ?
Thanks for your help

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Dec 28, 2016

@PVince81 how can I support you with more material for your analysis? (Or would you like to take a look into the full database?) Or how to step further on this issue?

@mirekys

This comment has been minimized.

Copy link
Contributor

mirekys commented Jan 2, 2017

Hi, I can confirm that this issue is present even in OC 8.2. I have seen the following SQL
exception when DELETE of the file fails for the first time:

"app":"webdav","message":"Exception:
{\"Message\":\"An exception occurred while executing 'UPDATE \\\"oc_filecache\\\" SET \\\"storage\\\" =  ?, \\\"path\\\" = ?, \\\"path_hash\\\" = ?, \\\"name\\\" = ?, \\\"parent\\\" =? WHERE \\\"fileid\\\" = ?' with params
[7572, \\\"files_trashbin\\\\\\\/files\\\\\\\/obfuscated.d1483099861\\\", \\\"974a8e7377fd148b0d042606a16db537\\\", \\\"obfuscated.d1483099861\\\", 147658637, 158461934]:
SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint \\\"fs_storage_path_hash\\\"
DETAIL:  Key (storage, path_hash)=(7572, 974a8e7377fd148b0d042606a16db537) already exists.\",
\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\UniqueConstraintViolationException\",\"Code\":0,\"Trace\":\"
#0 \\\/owncloud\\\/web\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(116): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))
#1 \\\/owncloud\\\/web\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Statement.php(172): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'UPDATE \\\"oc_file...', Array)
#2 \\\/owncloud\\\/web\\\/lib\\\/private\\\/db\\\/statementwrapper.php(73): Doctrine\\\\DBAL\\\\Statement->execute(Array)
#3 \\\/owncloud\\\/web\\\/lib\\\/private\\\/db.php(147): OC_DB_StatementWrapper->execute(Array)
#4 \\\/owncloud\\\/web\\\/lib\\\/private\\\/files\\\/cache\\\/cache.php(500): OC_DB::executeAudited('UPDATE `*PREFIX...', Array)
#5 \\\/owncloud\\\/web\\\/lib\\\/private\\\/files\\\/cache\\\/cache.php(447): OC\\\\Files\\\\Cache\\\\Cache->moveFromCache(Object(OC\\\\Files\\\\Cache\\\\HomeCache), 'files\\\/P...', 'files_trashbin\\\/...')
#6 \\\/owncloud\\\/web\\\/lib\\\/private\\\/files\\\/cache\\\/updater.php(198): OC\\\\Files\\\\Cache\\\\Cache->move('files\\\/P...', 'files_trashbin\\\/...')
#7 \\\/owncloud\\\/web\\\/apps\\\/files_trashbin\\\/lib\\\/trashbin.php(225): OC\\\\Files\\\\Cache\\\\Updater->rename('\\\/files\\\/...', '\\\/files_trashbin...')
#8 \\\/owncloud\\\/web\\\/apps\\\/files_trashbin\\\/lib\\\/storage.php(160): OCA\\\\Files_Trashbin\\\\Trashbin::move2trash('Pro/...')
#9 \\\/owncloud\\\/web\\\/apps\\\/files_trashbin\\\/lib\\\/storage.php(95): OCA\\\\Files_Trashbin\\\\Storage->doDelete('files\\\/P...', 'unlink')
#10 \\\/owncloud\\\/web\\\/lib\\\/private\\\/files\\\/view.php(1024): OCA\\\\Files_Trashbin\\\\Storage->unlink('files\\\/P...')
#11 \\\/owncloud\\\/web\\\/lib\\\/private\\\/files\\\/view.php(605): OC\\\\Files\\\\View->basicOperation('unlink', '\\\/Pro...', Array)
#12 \\\/owncloud\\\/web\\\/lib\\\/private\\\/connector\\\/sabre\\\/file.php(313): OC\\\\Files\\\\View->unlink('/Pro...')
#13 \\\/owncloud\\\/web\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(179): OC\\\\Connector\\\\Sabre\\\\File->delete()
#14 \\\/owncloud\\\/web\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(287): Sabre\\\\DAV\\\\Tree->delete('\\\/Pro...')
#15 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpDelete(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))
#16 \\\/owncloud\\\/web\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)
#17 \\\/owncloud\\\/web\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(469): Sabre\\\\Event\\\\EventEmitter->emit('method:DELETE', Array)
#18 \\\/owncloud\\\/web\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))
#19 \\\/owncloud\\\/web\\\/apps\\\/files\\\/appinfo\\\/remote.php(56): Sabre\\\\DAV\\\\Server->exec()
#20 \\\/owncloud\\\/web\\\/remote.php(137): require_once('\\\/owncloud\\\/...')
#21 {main}\",\"File\":\"\\\/owncloud\\\/web\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":63}",
"level":4,"time":"2016-12-30T13:11:01+01:00","method":"DELETE","url":"\/remote.php\/webdav\/Pro\/ofuscated"}

Relevant apache logs:

[30/Dec/2016:13:11:01 +0100] "DELETE /remote.php/webdav/Pro/obfuscated HTTP/1.1" 500 749 "-" "Mozilla/5.0 (Windows) mirall/2.2.4 (build 6408)" 0 1133 1664 312365
[30/Dec/2016:13:11:01 +0100] "DELETE /remote.php/webdav/Pro/My%20dir/obfuscated HTTP/1.1" 204 - "-" "Mozilla/5.0 (Windows) mirall/2.2.4 (build 6408)" 1 557 763 482456
[30/Dec/2016:13:12:00 +0100] "DELETE /remote.php/webdav/Pro/obfuscated HTTP/1.1" 403 193 "-" "Mozilla/5.0 (Windows) mirall/2.2.4 (build 6408)" 4 544 992 259291

Directory listing on web frontend logs that 'Could not locate file exception'. Clients trying to
delete the file do always receive a 403 Forbidden exception. Filecache contains only
one single entry for the file:

owncloud=# SELECT * FROM oc_filecache WHERE name LIKE '%obfuscated%';
  fileid   | storage |                     path                      |            path_hash             |  parent   |      name        | mimetype | mimepart |  size  |   mtime    | storage_mtime | encrypted | unencrypted_size |               etag               | permissions 
-----------+---------+-----------------------------------------------+----------------------------------+-----------+------------------+----------+----------+--------+------------+---------------+-----------+------------------+----------------------------------+-------------
 158461934 |    7572 | files/Pro/obfuscated | c029d986c1557b8aa735e2935ecaf930 | 147407588 | obfuscated |       10 |        3 | 305958 | 1483099173 |    1483099173 |         0 |                0 | 53ece7a34b42d8eac5ebca339d676b71 |          27

It doesn't exist anywhere on the filesystem (even in files_trashbin).
Haven't tried rescan of files yet, but will try it soon.
Possibly could also be related to #23151 .

@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Jan 9, 2017

I need to find time to try and reproduce this issue locally with the provided steps...

@mirekys @esprit-libre can you confirm that your issue is 100% the same as @JochenHWezel ? Do you also have the app "working2gether" enabled ? The more info about your setups, the more we can cross-relate similarities and find out why this is happening.

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Jan 9, 2017

I confirm that I had the same issue, with workin2gether enabled. You have my config detail above.
I can also indicate that manually cleaning oc_filecache solve the issue. I don't have new use case since the one I declared.

@mirekys

This comment has been minimized.

Copy link
Contributor

mirekys commented Jan 11, 2017

@PVince81 yes, the issue is pretty much the same. The "working2gether" app was never present in mine environment, though. Manual rescan of user's files helped to resolve the issue for me.
My setup is following:

Server configuration

Operating system:
RHEL 6.4

Web server:
httpd 2.4

Database:
PostgreSQL 9.2

PHP version:
5.6

ownCloud version:
8.2.5

Updated from an older ownCloud or fresh install:
Updated from 8.1.x

Where did you install ownCloud from:
Official tarball

List of activated apps:

Enabled:
  - activity: 2.1.4
  - calendar: 0.8.2
  - contacts: 0.5.0.0
  - documents: 0.11.1
  - files: 1.2.1
  - files_pdfviewer: 0.7.1
  - files_sharing: 0.7.0
  - files_texteditor: 2.0
  - files_trashbin: 0.7.0
  - files_versions: 1.1.0
  - firstrunwizard: 1.1
  - notifications: 0.1.0
  - user_shib: 0.0.1

The content of config/config.php:

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "8.2.5.2",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "forcessl": true,
        "maintenance": false,
        "enable_previews": false,
        "singleuser": false,
        "secret": "***REMOVED SENSITIVE VALUE***",
        "share_folder": "\/Shared",
        "trashbin_retention_obligation": "auto",
        "mail_smtpmode": "smtp",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "\/tmp\/redis\/redis.sock",
            "port": 0
        },
        "allow_user_to_change_display_name": true,
        "session_lifetime": xxx,
        "session_keepalive": true,
        "asset-pipeline.enabled": true
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
no

Are you using encryption: yes/no
no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
user_shib

@mirekys

This comment has been minimized.

Copy link
Contributor

mirekys commented Jan 19, 2017

@JochenHWezel @esprit-libre @PVince81 could you please have a look at that #23151 issue if you can reproduce that in your environment? I still believe that it is very closely related to this issue and it is quite easy to reproduce for me (and, unfortunately, the users :)) as described here: #23151 (comment) .

@esprit-libre

This comment has been minimized.

Copy link

esprit-libre commented Feb 1, 2017

@mirekys looks like a similar problem. I will try to reproduce.

@PVince81 I have the problem for the 3rd time on a 3rd instance... every time with 9.1.3 version, and every time it happens a while after upgrade to 9.1.3 with big folder deletion.
Problem could be linked to client issue 5509?
This time, no use of workin2gether, so not involved in the problem.

@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Apr 6, 2017

If moving big folder, it might be this issue: #25569.
Then when trying to delete a file from the inconsistency, you'll get a 404.

Starting with 10.0 if a file has a filecache entry but doesn't exist on disk, it will still delete it to avoid these useless errors.

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Jun 18, 2017

after upgrading to OwnCloud v10.0.2, I re-tried to delete those file entries

  1. Trial: using a file scan on server succeeded, but didn't fix any file entries for missing files

Scanning files for 17 users
Starting scan for user 1 out of 17 (...)
Starting scan for user 2 out of 17 (...)
Starting scan for user 3 out of 17 (...)
Starting scan for user 4 out of 17 (...)
Starting scan for user 5 out of 17 (...)
Starting scan for user 6 out of 17 (...)
Starting scan for user 7 out of 17 (...)
...
Starting scan for user 16 out of 17 (...)
Starting scan for user 17 out of 17 (...)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5130    | 41923 | 00:20:23     |
+---------+-------+--------------+
  1. Trial: deleting the file entries using the web frontend
    image
    same procedure as several months ago: the file entry is deleted at frontend, first. But when I reopen the folder, the file system entry is back again...

@PVince81 PVince81 added this to the triage milestone Jun 19, 2017

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Aug 27, 2017

I've found 1 way how this whole issue came to happen:

  1. for whatever reason, a file had sync errors (oc sync client displayed tag mismatch errors) - to the same time this shared file was visible in web UI 2 times(!) with the very same file name in the very same folder
  2. I moved that file on a local machine into another folder and synced this change with the oc sync client
  3. The file appeared on the web UI in the new folder plus in the old folder
  4. The oc sync client shows for the file in the old folder the error message that the file has been removed from server

The file can be deleted in web front end from old location, BUT it re-appears again after reload in web front end (effectively, delete doesn't work).

An occ files:scan --all doesn't solve this issue. The oc sync client and the web front end still show the file / try to download the non-existing file.

The old and wrong file entry doesn't appear in SQL table oc_filecache:

mysql> select * from oc_filecache where path like '%.psafe3' and name = 'PWSafe BEI-IT.psafe3' limit 0, 100;
+--------+---------+----------------------------------------------------------------+----------------------------------+--------+----------------------+----------+----------+-------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
| fileid | storage | path                                                           | path_hash                        | parent | name                 | mimetype | mimepart | size  | mtime      | storage_mtime | encrypted | unencrypted_size | etag                             | permissions | checksum |
+--------+---------+----------------------------------------------------------------+----------------------------------+--------+----------------------+----------+----------+-------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
|  15825 |       1 | files_trashbin/files/Security.d1466382859/PWSafe BEI-IT.psafe3 | ba81da666052891bb3ef48b9741af368 |  15824 | PWSafe BEI-IT.psafe3 |        7 |        5 | 10776 | 1465755236 |    1465755236 |         0 |                0 | 86a1afe32b990dc4b688d0aaff2cae61 |          27 |          |
|  17968 |       1 | files/BEI IT-Administration/Security/PWSafe BEI-IT.psafe3      | d967bd6bfcc323671a84ad9f13dfa0bb |  17967 | PWSafe BEI-IT.psafe3 |        7 |        5 | 11032 | 1487276438 |    1487276438 |         0 |                0 | e5d165db1ca20ede59c1135e876498a9 |          27 |          |
|  70852 |       1 | files/BEI IT-Security/PasswordSafes/Temp/PWSafe BEI-IT.psafe3  | 6bd9af0fd6f69f00d24d21eac38e3892 |  70850 | PWSafe BEI-IT.psafe3 |        7 |        5 | 11208 | 1503245253 |    1503245253 |         0 |                0 | ede445a458cea4708e5aa24f3b05899f |          27 |          |
+--------+---------+----------------------------------------------------------------+----------------------------------+--------+----------------------+----------+----------+-------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
3 rows in set (0,11 sec)

(but the new file location entry in the subfolder "Temp" is available in the table)

ATTENTION: there seems to be a file in "files/BEI IT-Administration/Security/PWSafe BEI-IT.psafe3". But the folder "files/BEI IT-Administration/Security" doesn't exist any more on this place - it was moved to files/BEI IT-Security" several months ago...

QUESTION: In which table I have to search for the file entry which should not exist any more but still is accessed by the oc sync client and web front end to display this file?
image

@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Aug 28, 2017

you could try running the SQL queries from the first post in #28253, maybe you got a "filecache hole":

@JochenHWezel

This comment has been minimized.

Copy link
Author

JochenHWezel commented Sep 24, 2017

select fc.storage,fc.fileid,fc.parent as "wrongparent",fc3.fileid "correctparent",fc.path,fc.etag from oc_filecache fc, oc_filecache fc3 where fc.parent <> -1 and fc.parent != fc3.fileid and fc3.storage=fc.storage and fc3.path = substring(fc.path, 1, length(fc.path) - length(substring_index(fc.path, '/', -1)) - 1) and not exists (select 1 from oc_filecache fc2 where fc2.fileid = fc.parent) order by path;

Empty set (0,11 sec)

SELECT fc.storage, fc.fileid, fc.path as path, fc.name, fcp.storage as parentstorage, fcp.path as parentpath FROM oc_filecache fc, oc_filecache fcp WHERE (fc.parent = fcp.fileid) AND ( (CONCAT(fcp.path, '/', fc.name) <> fc.path) OR (fc.storage <> fcp.storage) ) AND (fcp.path <> '') AND (fc.fileid <> fcp.fileid);

{see below}

select fc.storage,fc.fileid,fc.parent as "wrongparent",fc.path,fc.etag from oc_filecache fc where fc.parent <> -1 and not exists (select 1 from oc_filecache fc2 where fc2.fileid = fc.parent) order by path;`

Empty set (0,30 sec)

select * from oc_filecache where parent=fileid;`

Empty set (0,11 sec)

mysql> SELECT fc.`storage`, fc.`fileid`, fc.`path` as `path`, fc.`name`, fcp.`storage` as `parentstorage`, fcp.`path` as `parentpath`     FROM `oc_filecache` fc, `oc_filecache` fcp     WHERE (fc.`parent` = fcp.`fileid`)     AND (         (CONCAT(fcp.`path`, '/', fc.`name`) <> fc.`path`)         OR (fc.`storage` <> fcp.`storage`)     )     AND (fcp.`path` <> '')     AND (fc.`fileid` <> fcp.`fileid`);
+---------+--------+----------------------------------------------------------------------------------------------------------------+---------------------------------------------------+---------------+------------------------------------------------------------------------------------------------------------------+
| storage | fileid | path                                                                                                           | name                                              | parentstorage | parentpath                                                                                                       |
+---------+--------+----------------------------------------------------------------------------------------------------------------+---------------------------------------------------+---------------+------------------------------------------------------------------------------------------------------------------+
|       1 |   5173 | files/BEI IT-Administration/Security/BEI Netzwerk.ibak                                                         | BEI Netzwerk.ibak                                 |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  17998 | files/BEI IT-Administration/Security/PWSafe BEI-IT.ibak                                                        | PWSafe BEI-IT.ibak                                |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  18040 | files/BEI IT-Administration/Security/PWSafe BEI-IT_002.ibak                                                    | PWSafe BEI-IT_002.ibak                            |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  18045 | files/BEI IT-Administration/Security/PWSafe BEI-IT_003.ibak                                                    | PWSafe BEI-IT_003.ibak                            |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  21420 | files/BEI IT-Administration/Security/PWSafe BEI-IT_004.ibak                                                    | PWSafe BEI-IT_004.ibak                            |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  27291 | files/BEI IT-Administration/Security/BEI Netzwerk_011.ibak                                                     | BEI Netzwerk_011.ibak                             |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  27295 | files/BEI IT-Administration/Security/BEI Netzwerk_012.ibak                                                     | BEI Netzwerk_012.ibak                             |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  60768 | files/BEI IT-Administration/Security/BEI Netzwerk_013.ibak                                                     | BEI Netzwerk_013.ibak                             |             1 | files/BEI IT-Security/PasswordSafes                                                                              |
|       1 |  69718 | files/BEI Technik/2017-08-12 Hochzeit Sam+Sarah/170525SimonZ0973.jpg                                           | 170525SimonZ0973.jpg                              |             1 | files/BEI Bilder & Videos/2017-08-12 Hochzeit Sam+Sarah                                                          |
|       1 |  70144 | files/BEI Technik/2017-08-12 Hochzeit Sam+Sarah/170525SimonZ0696.jpg                                           | 170525SimonZ0696.jpg                              |             1 | files/BEI Bilder & Videos/2017-08-12 Hochzeit Sam+Sarah                                                          |
[...]
+---------+--------+----------------------------------------------------------------------------------------------------------------+---------------------------------------------------+---------------+------------------------------------------------------------------------------------------------------------------+
308 rows in set (0,15 sec)
@PVince81

This comment has been minimized.

Copy link
Member

PVince81 commented Oct 9, 2017

Ok, so you got mismatched parents.

To fix it you'll either need to wait for 10.0.4 or if you want to try it locally:

  1. backup the database!
  2. apply #29074 (comment)
  3. find the user id of affected users for the storages found in the results above
  4. run occ files:scan $userId --repair. Replace $userId with the id of the affected user

or

  1. backup the database!
  2. apply #29074 (comment)
  3. switch the instance to single user mode: occ maintenance:singleuser --on
  4. run occ files:scan --all --repair
  5. switch the instance back: occ maintenance:singleuser --off

Note that the fix in question isn't released yet but has been manually tested on several instances and reported working fine.

@ownclouders

This comment has been minimized.

Copy link

ownclouders commented Dec 13, 2017

Hey, this issue has been closed because the label needs info is set and there were no updates for 14 days. Feel free to reopen this issue if you deem it appropriate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment