Query the cache when checking if a node exists #22506

icewind1991 · 2016-02-18T14:42:40Z

Improves performance when working with slow storage backends

mention-bot · 2016-02-18T14:42:42Z

By analyzing the blame information on this pull request, we identified @MorrisJobke, @PVince81 and @nickvergessen to be potential reviewers

rullzer · 2016-02-19T06:48:19Z

Failing unit tests

icewind1991 · 2016-02-19T14:38:22Z

Failing unit tests

All fixed

nickvergessen · 2016-02-19T15:27:40Z

lib/base.php

@@ -129,7 +129,12 @@ public static function initPaths() {
 		} else {
 			self::$configDir = OC::$SERVERROOT . '/config/';
 		}
-		self::$config = new \OC\Config(self::$configDir);
+
+		if (defined('PHPUNIT_RUN') and PHPUNIT_RUN) {


nickvergessen · 2016-02-19T15:28:47Z

👎 for the hack in base.php

icewind1991 · 2016-02-22T10:18:11Z

Switched to mocking the config

I would still like to add a system to automatically handle it for all tests in the future

icewind1991 · 2016-02-22T17:09:11Z

@nickvergessen @rullzer @PVince81 please review

rullzer · 2016-02-25T10:56:54Z

👍

MorrisJobke · 2016-02-25T17:26:37Z

So this should really go into 9.0?

cc @karlitschek @cmonteroluque

karlitschek · 2016-02-25T17:31:06Z

Hmmm. relatively risky after feature freeze and 3 days before the freeze. What do you think @cmonteroluque ?

PVince81 · 2016-02-25T17:38:55Z

This did not help fixing the original issue anyway, so I'd say 9.0.1

PVince81 · 2016-02-29T13:22:28Z

Moving to 9.0.1 as this doesn't fix any critical known issue

CC @cmonteroluque

ghost · 2016-03-03T18:37:09Z

agreed

LukasReschke · 2016-03-08T17:50:09Z

lib/private/files/node/folder.php

@@ -100,9 +100,9 @@ protected function createNode($path, $info = array()) {
 			$isDir = $info['mimetype'] === 'httpd/unix-directory';
 		}
 		if ($isDir) {
-			return new Folder($this->root, $this->view, $path);
+			return new Folder($this->root, $this->view, $path, $info);


$info seems to be an array as per PHPDoc. Folder seems however to expect \OCP\Files\FileInfo. Could you adjust this so that static code analysers are happy?

MorrisJobke · 2016-03-18T08:38:03Z

@icewind1991 Could you checkout the above comments?

icewind1991 · 2016-03-21T13:22:12Z

switched everything to FileInfo objects

icewind1991 · 2016-03-22T13:35:13Z

@LukasReschke @MorrisJobke @PVince81 please review

PVince81 · 2016-03-22T14:24:05Z

👍

PVince81 · 2016-03-22T19:27:30Z

@nickvergessen @rullzer rereview after the changes ?

MorrisJobke · 2016-03-22T21:41:10Z

Is the 9.0.1 milestone indented? I would rather not backport this and only merge it to master.

rullzer · 2016-03-23T07:14:44Z

PR is against mater so goes into 9.1 anyway...

MorrisJobke · 2016-03-23T11:53:47Z

Tested and works 👍

icewind1991 · 2016-03-23T14:31:30Z

#22463 is set for 9.0.1 which this is supposed to fix, should this be backported ( @karlitschek )

PVince81 · 2016-03-23T14:33:01Z

@icewind1991 this PR doesn't fix the underlying problem, so for now I'd rather say no backport.

icewind1991 · 2016-03-23T15:00:59Z

There are 2 parts of #22463

node api using the storage directly where it can use the cache which makes it slow on external storages
- tested this using an artificial slowdown on storages
- fixed with this pr
repeated scanning of storages making everything slow
- haven't found any reliable way to reproduce this and I don't know the cause

PVince81 · 2016-03-23T15:04:56Z

Ok fair enough.

@karlitschek backport to 9.0.1 ?

karlitschek · 2016-03-23T16:05:15Z

great. please backport

icewind1991 · 2016-03-23T16:56:51Z

stable9: #23520

lock · 2019-08-06T19:01:40Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

icewind1991 added the 3 - To Review label Feb 18, 2016

icewind1991 added this to the 9.0-current milestone Feb 18, 2016

DeepDiver1975 added the in progress label Feb 18, 2016

icewind1991 mentioned this pull request Feb 18, 2016

OCS Share API very slow with slow external storage #22463

Closed

nickvergessen added the comp:filesystem label Feb 19, 2016

icewind1991 force-pushed the node-get-from-cache branch 3 times, most recently from 0f3fda1 to d1e747b Compare February 19, 2016 13:29

nickvergessen reviewed Feb 19, 2016
View reviewed changes

icewind1991 force-pushed the node-get-from-cache branch from d1e747b to 9d38091 Compare February 22, 2016 10:17

icewind1991 force-pushed the node-get-from-cache branch from 9d38091 to 6e4cc60 Compare February 22, 2016 14:38

rullzer added the performance label Feb 25, 2016

PVince81 modified the milestones: 9.0.1-next-maintenance, 9.0-current Feb 29, 2016

LukasReschke reviewed Mar 8, 2016
View reviewed changes

icewind1991 added 3 commits March 21, 2016 13:53

improve reuse in getUserFolder

6031ae1

set watch policy in test

d0dd76b

properly use fileinfo objects

0b0b325

icewind1991 force-pushed the node-get-from-cache branch from 6e4cc60 to 0b0b325 Compare March 21, 2016 13:21

rullzer modified the milestones: 9.1-current, 9.0.1-current-maintenance Mar 23, 2016

MorrisJobke added 4 - To release and removed 3 - To Review labels Mar 23, 2016

DeepDiver1975 merged commit ea07a42 into master Mar 23, 2016

DeepDiver1975 deleted the node-get-from-cache branch March 23, 2016 12:08

DeepDiver1975 removed the in progress label Mar 23, 2016

PVince81 added the backport-request label Mar 23, 2016

icewind1991 mentioned this pull request Mar 23, 2016

[9.0] Query the cache when checking if a node exists #23520

Merged

MorrisJobke added 9.0.1-current and removed backport-request labels Mar 24, 2016

lock bot locked as resolved and limited conversation to collaborators Aug 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Query the cache when checking if a node exists #22506

Query the cache when checking if a node exists #22506

icewind1991 commented Feb 18, 2016

mention-bot commented Feb 18, 2016

rullzer commented Feb 19, 2016

icewind1991 commented Feb 19, 2016

nickvergessen Feb 19, 2016

nickvergessen commented Feb 19, 2016

icewind1991 commented Feb 22, 2016

icewind1991 commented Feb 22, 2016

rullzer commented Feb 25, 2016

MorrisJobke commented Feb 25, 2016

karlitschek commented Feb 25, 2016

PVince81 commented Feb 25, 2016

PVince81 commented Feb 29, 2016

ghost commented Mar 3, 2016

LukasReschke Mar 8, 2016

MorrisJobke commented Mar 18, 2016

icewind1991 commented Mar 21, 2016

icewind1991 commented Mar 22, 2016

PVince81 commented Mar 22, 2016

PVince81 commented Mar 22, 2016

MorrisJobke commented Mar 22, 2016

rullzer commented Mar 23, 2016

MorrisJobke commented Mar 23, 2016

icewind1991 commented Mar 23, 2016

PVince81 commented Mar 23, 2016

icewind1991 commented Mar 23, 2016

PVince81 commented Mar 23, 2016

karlitschek commented Mar 23, 2016

icewind1991 commented Mar 23, 2016

lock bot commented Aug 6, 2019

Query the cache when checking if a node exists #22506

Query the cache when checking if a node exists #22506

Conversation

icewind1991 commented Feb 18, 2016

mention-bot commented Feb 18, 2016

rullzer commented Feb 19, 2016

icewind1991 commented Feb 19, 2016

nickvergessen Feb 19, 2016

Choose a reason for hiding this comment

nickvergessen commented Feb 19, 2016

icewind1991 commented Feb 22, 2016

icewind1991 commented Feb 22, 2016

rullzer commented Feb 25, 2016

MorrisJobke commented Feb 25, 2016

karlitschek commented Feb 25, 2016

PVince81 commented Feb 25, 2016

PVince81 commented Feb 29, 2016

ghost commented Mar 3, 2016

LukasReschke Mar 8, 2016

Choose a reason for hiding this comment

MorrisJobke commented Mar 18, 2016

icewind1991 commented Mar 21, 2016

icewind1991 commented Mar 22, 2016

PVince81 commented Mar 22, 2016

PVince81 commented Mar 22, 2016

MorrisJobke commented Mar 22, 2016

rullzer commented Mar 23, 2016

MorrisJobke commented Mar 23, 2016

icewind1991 commented Mar 23, 2016

PVince81 commented Mar 23, 2016

icewind1991 commented Mar 23, 2016

PVince81 commented Mar 23, 2016

karlitschek commented Mar 23, 2016

icewind1991 commented Mar 23, 2016

lock bot commented Aug 6, 2019