File lock prevents file scan on external webdav storage

[jfeldmaier]

Steps to reproduce

1. Nextcloud setup with external storage on IONOS HiDrive Storage (attached via webdav)
2. Some files stored locally on the disk where the server is also installed
3. Try to do a file scan for all users: sudo -u www-data php occ files:scan --all

Expected behaviour

The file scan should scan internal as well as the external files.

Actual behaviour

The file scan stops after scanning some of the locally stored files and reports an error on an external file:

Starting scan for user 1 out of 2 (xxx)
Exception during scan: "files/93f47f2219cd30e07a85b622ab4xxxxxx"("webdav::someuser1234@webdav.hidrive.ionos.com///users/someuser1234/::scanner::") is locked
#0 /var/www/html/cloud/lib/private/Files/Storage/Common.php(751): OC\Lock\DBLockingProvider->acquireLock()
#1 /var/www/html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php(594): OC\Files\Storage\Common->acquireLock()
#2 /var/www/html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php(594): OC\Files\Storage\Wrapper\Wrapper->acquireLock()
#3 /var/www/html/cloud/lib/private/Files/Cache/Scanner.php(333): OC\Files\Storage\Wrapper\Wrapper->acquireLock()
#4 /var/www/html/cloud/lib/private/Files/Utils/Scanner.php(260): OC\Files\Cache\Scanner->scan()
#5 /var/www/html/cloud/apps/files/lib/Command/Scan.php(152): OC\Files\Utils\Scanner->scan()
#6 /var/www/html/cloud/apps/files/lib/Command/Scan.php(208): OCA\Files\Command\Scan->scanFiles()
#7 /var/www/html/cloud/apps/mail/vendor/symfony/console/Command/Command.php(258): OCA\Files\Command\Scan->execute()
#8 /var/www/html/cloud/core/Command/Base.php(169): Symfony\Component\Console\Command\Command->run()
#9 /var/www/html/cloud/apps/mail/vendor/symfony/console/Application.php(920): OC\Core\Command\Base->run()
#10 /var/www/html/cloud/apps/mail/vendor/symfony/console/Application.php(266): Symfony\Component\Console\Application->doRunCommand()
#11 /var/www/html/cloud/apps/mail/vendor/symfony/console/Application.php(142): Symfony\Component\Console\Application->doRun()
#12 /var/www/html/cloud/lib/private/Console/Application.php(215): Symfony\Component\Console\Application->run()
#13 /var/www/html/cloud/console.php(100): OC\Console\Application->run()
#14 /var/www/html/cloud/occ(11): require_once('/var/www/html/c...')
#15 {main}

Server configuration

Operating system:
Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Web server:
Server version: Apache/2.4.41 (Ubuntu)
Server built: 2020-08-12T19:46:17

Database:
Server version: 10.3.25-MariaDB-0ubuntu0.20.04.1 Ubuntu 20.04

PHP version:
PHP 7.4.3 (cli) (built: Oct 6 2020 15:47:56) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Zend OPcache v7.4.3, Copyright (c), by Zend Technologies

Nextcloud version: (see Nextcloud admin page)
Nextcloud 20.0.5

Updated from an older Nextcloud/ownCloud or fresh install:
Started at Nextcloud 15 and updated over the "years" to 20.0.5

Where did you install Nextcloud from:
From download page and via updater

Signing status:
No errors have been found.

Signing status

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

List of activated apps:

App list

Enabled:
  - accessibility: 1.6.0
  - activity: 2.13.4
  - cloud_federation_api: 1.3.0
  - comments: 1.10.0
  - contactsinteraction: 1.1.0
  - dav: 1.16.2
  - federatedfilesharing: 1.10.2
  - federation: 1.10.1
  - files: 1.15.0
  - files_external: 1.11.1
  - files_pdfviewer: 2.0.1
  - files_rightclick: 0.17.0
  - files_sharing: 1.12.2
  - files_trashbin: 1.10.1
  - files_versions: 1.13.0
  - files_videoplayer: 1.9.0
  - firstrunwizard: 2.9.0
  - logreader: 2.5.0
  - lookup_server_connector: 1.8.0
  - mail: 1.7.2
  - nextcloud_announcements: 1.9.0
  - notifications: 2.8.0
  - oauth2: 1.8.0
  - password_policy: 1.10.1
  - photos: 1.2.3
  - privacy: 1.4.0
  - provisioning_api: 1.10.0
  - recommendations: 0.8.0
  - serverinfo: 1.10.0
  - settings: 1.2.0
  - sharebymail: 1.10.0
  - support: 1.3.0
  - survey_client: 1.8.0
  - systemtags: 1.10.0
  - text: 3.1.0
  - theming: 1.11.0
  - twofactor_backupcodes: 1.9.0
  - updatenotification: 1.10.0
  - user_status: 1.0.1
  - viewer: 1.4.0
  - weather_status: 1.0.0
  - workflowengine: 2.2.0
Disabled:
  - admin_audit
  - calendar
  - dashboard
  - encryption
  - user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.xxx.de"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "20.0.5.2",
        "overwrite.cli.url": "https:\/\/cloud.xxx.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "maintenance": false,
        "updater.secret": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "loglevel": 0,
        "mysql.utf8mb4": true
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
webdav on IONOS hidrive

Are you using encryption: yes/no
no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

LDAP configuration (delete this part if not used)

LDAP config

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your Nextcloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser:
Chrome
Operating system:

Logs

Web server error log

Web server error log

[Thu Feb 11 00:00:13.626219 2021] [core:notice] [pid 854] AH00094: Command line: '/usr/sbin/apache2'
[Thu Feb 11 00:31:30.328489 2021] [php7:error] [pid 22689] [client 45.155.205.108:47368] script '/var/www/html/index.php' not found or unable to stat
[Thu Feb 11 05:50:28.363883 2021] [negotiation:error] [pid 22803] [client 45.155.205.108:34962] AH00687: Negotiation: discovered file(s) matching request: /var/www/html/cloud/console (None could be negotiated).
[Thu Feb 11 08:10:12.533052 2021] [php7:error] [pid 22990] [client 123.206.205.78:37604] script '/var/www/html/elrekt.php' not found or unable to stat
[Thu Feb 11 08:10:13.010054 2021] [php7:error] [pid 22689] [client 123.206.205.78:37850] script '/var/www/html/index.php' not found or unable to stat
[Thu Feb 11 17:39:31.457095 2021] [php7:error] [pid 22803] [client 45.155.205.108:54848] script '/var/www/html/index.php' not found or unable to stat
[Thu Feb 11 22:06:56.411540 2021] [access_compat:error] [pid 30262] [client 91.2.56.108:50074] AH01797: client denied by server configuration: /var/www/html/cloud/data/.ocdata
[Thu Feb 11 22:07:44.039025 2021] [access_compat:error] [pid 28508] [client 91.2.56.108:50092] AH01797: client denied by server configuration: /var/www/html/cloud/data/.ocdata

Nextcloud log (data/nextcloud.log)

Nextcloud log

I removed the log file shortly before starting the scan:

{"reqId":"4z8SGd5wgB0DyhcDdEb5","level":0,"time":"2021-02-11T21:29:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_external","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"}
{"reqId":"4z8SGd5wgB0DyhcDdEb5","level":0,"time":"2021-02-11T21:29:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_sharing","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"}
{"reqId":"afOJ5fVCJbtx4ieCt8WI","level":0,"time":"2021-02-11T21:29:43+00:00","remoteAddr":"","user":"--","app":"files_external","method":"","url":"--","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"20.0.5.2"}
{"reqId":"afOJ5fVCJbtx4ieCt8WI","level":0,"time":"2021-02-11T21:29:43+00:00","remoteAddr":"","user":"--","app":"files_sharing","method":"","url":"--","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"20.0.5.2"}
{"reqId":"uXoRZwLdGi0XXatmfwTn","level":0,"time":"2021-02-11T21:30:07+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_external","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"}
{"reqId":"uXoRZwLdGi0XXatmfwTn","level":0,"time":"2021-02-11T21:30:07+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_sharing","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"}
{"reqId":"qSFDBAEzeNGOipvuuzkT","level":0,"time":"2021-02-11T21:30:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_external","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"}
{"reqId":"qSFDBAEzeNGOipvuuzkT","level":0,"time":"2021-02-11T21:30:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_sharing","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"}

Please let me know if I should produce some other debug information!

[szaimen]

Is this Issue still valid in NC21.0.2? If not, please close this issue. Thanks! :)

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.