Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Very slow access (copy, ls, etc) files in folders with lots of files through webdav + proposed fix #8962
Access (copy, ls, etc) files from folders with thousands of files through webdav is very slow (10-20 or more seconds for simple
(/home/sketis/docs.mini-maxi.lt/ is mounted through davfs (davfs2 package), folder /home/sketis/docs.mini-maxi.lt/2_Nuotrauku_katalogas/imported/ contains ~9000 files size 100-600kb)
I've tried lots of optimizations, including redis and database buffer, etc optimizations and almost nothing helps here, but after 3 days of hard work I've found where is the main issue:
every operation with any file from folder with thousands of files through webdav protocol generates thousands queries (one or 2 query for every file from that folder):
This is a bug, because I access (for example copy) only one file, not all files from folder. Operation takes 2x longer, when I try to access file, which doesn't exists:
I've activated "pg_stat_statements" module in PostgreSQL and then found the main issue:
67532 queries just after ~2 minutes! "oc_properties" table contains ~12000 rows.
So, there is one simple workaround - I've increased query speed 10 times by creating and index in oc_properties table for both columns - userid and propertypath:
CREATE INDEX properties_path_index ON oc_properties USING btree (userid, propertypath);
Please add this index in Nexcloud database creation file.
But index creation only partially solves the performance issues with lots of files in one folder - it would be wise to create the setting "Ignore executable bit in WEBDAV" in nextcloud's config.php and don't query oc_properties table at all when this setting is set to true.
Steps to reproduce
ls, cp or rm one or several files from folder should take 1 or 2 seconds
ls, cp or rm one or several files from folder takes 10-40 seconds
Operating system: Ubuntu 16.04.4 64-bit
Web server: apache (also tested with nginx - almost identical performance)
PHP version: 7.0.28
Nextcloud version: 13.0.1 (also tested with all 12.x versions)
Updated from an older Nextcloud/ownCloud or fresh install: Updated from 12.x
Where did you install Nextcloud from: from nextcloud.org downloads
Config report'ocy77dpvulpn', 'passwordsalt' => 'good_password', 'secret' => 'no_secrets', 'trusted_domains' => array ( 0 => '192.168.199.53', ), 'datadirectory' => '/srv/nextcloud-duomenys', 'overwrite.cli.url' => 'https://192.168.199.53/', 'htaccess.RewriteBase' => '/', 'dbtype' => 'pgsql', 'version' => '18.104.22.168', 'dbname' => 'nextcloud', 'dbhost' => 'localhost', 'dbport' => '', 'dbtableprefix' => 'oc_', 'dbuser' => 'nextuser', 'dbpassword' => 'db-password', 'installed' => true, 'mail_smtpmode' => 'smtp', 'mail_smtpauthtype' => 'LOGIN', 'mail_smtpsecure' => 'ssl', 'mail_from_address' => 'docs', 'mail_domain' => 'my.mail.lt', 'mail_smtpauth' => 1, 'mail_smtphost' => 'my.mail..lt', 'mail_smtpport' => '465', 'mail_smtpname' => 'firstname.lastname@example.org', 'mail_smtppassword' => 'mail_password', 'mail_smtpdebug' => true, 'auth.bruteforce.protection.enabled' => false, 'maintenance' => false, 'theme' => '', 'loglevel' => 2, 'memcache.local' => '\\OC\\Memcache\\APCu', 'redis' => array ( 'host' => 'localhost', 'port' => 6379, ), );
Are you using external storage, if yes which one: local
Are you using encryption: no
Are you using an external user-backend, if yes which one: no
Operating system: Ubuntu 16.04 64-bit
referenced this issue
Mar 23, 2018
We had a look at this and adding the index there is not that easy as it might work fine in Postgres but for mysql there is some limitation on the index length in default setups. :/
So we will move this to 15 as there is not enough time left to look into this properly.
Something I've noticed is that it runs the "getScanner" method on the OC\Files\View class every file every time you do anything with the local storage. Even when I'm 3 or 4 folders deep, if I reload that page, it scans every folder from the root up to where I'm at.
I've been trying to profile the folder loading (trying to figure out which function is the culprit), and it looks like if I'm at a location like this
Then it scans every file/folder in "Local Storage Root", then "Brian Leishman", "Pictures", and finally "Food", so when "Local Storage Root" has 11k folders in it, they all get scanned again at this point, which means that accessing that last folder with 1 item in it takes about a minute to load (even on a very, very fast machine with all the performance tuning enabled).
I'm trying to fix this myself for our use case, but I'm getting stuck trying to figure out if it's the Sabre/DAV 3rd party tool or NextCloud itself