cache getStorageById #27312

Merged
merged 9 commits into from Mar 9, 2017

Conversation

Projects
None yet
5 participants
@butonic
Member

butonic commented Mar 6, 2017

Description

getStorageById is indirectly called by the Availability wrapper, easily leading to hundreds of db queries. With this PR we not only cache the result for the current request but also push it to a distributed cache for 5 min, relieving the DB from executing that query.

How Has This Been Tested?

Locally with a user_shibboleth user and 100 incoming shares from 100 different user_shibboleth users (each user shared a single folder).

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@butonic butonic added this to the 9.0.9 milestone Mar 6, 2017

@butonic butonic requested review from PVince81 and mrow4a Mar 6, 2017

@mention-bot

This comment has been minimized.

Show comment
Hide comment
@mention-bot

mention-bot Mar 6, 2017

@butonic, thanks for your PR! By analyzing the history of the files in this pull request, we identified @DeepDiver1975 and @PVince81 to be potential reviewers.

@butonic, thanks for your PR! By analyzing the history of the files in this pull request, we identified @DeepDiver1975 and @PVince81 to be potential reviewers.

@PVince81

The approach looks fine

lib/private/Files/Cache/Storage.php
@@ -181,6 +231,17 @@ public static function remove($storageId) {
$sql = 'DELETE FROM `*PREFIX*storages` WHERE `id` = ?';
\OC_DB::executeAudited($sql, [$storageId]);
+ // delete from local cache
+ if(self::$localCache) {

This comment has been minimized.

@PVince81

PVince81 Mar 6, 2017

Member

self::$localCache === null for consistency with below ?

@PVince81

PVince81 Mar 6, 2017

Member

self::$localCache === null for consistency with below ?

lib/private/Files/Cache/Storage.php
+ */
+ private static function getStorageByIdFromCache($storageId) {
+ if (self::$distributedCache === null) {
+ self::$distributedCache =

This comment has been minimized.

@PVince81

PVince81 Mar 6, 2017

Member

I guess this returns a NullCache if none was configured ?

@PVince81

PVince81 Mar 6, 2017

Member

I guess this returns a NullCache if none was configured ?

This comment has been minimized.

@butonic

butonic Mar 6, 2017

Member

yep

lib/private/Files/Cache/Storage.php
+ self::$localCache = new CappedMemoryCache();
+ }
+ $result = self::$localCache->get($storageId);
+ if (empty($result)) {

This comment has been minimized.

@PVince81

PVince81 Mar 6, 2017

Member

in PHP empty('0') is true, better be safe...

@PVince81

PVince81 Mar 6, 2017

Member

in PHP empty('0') is true, better be safe...

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 6, 2017

Member

@PVince81 thx for the quick review. I am noticing unneeded queries we could cache if also caching not existing storages. similar to the user database table.

There is a legacy check that tries to find an old local:: storage for every home:: storage ... causing an unnecessary query for every home storage initialized per request

Member

butonic commented Mar 6, 2017

@PVince81 thx for the quick review. I am noticing unneeded queries we could cache if also caching not existing storages. similar to the user database table.

There is a legacy check that tries to find an old local:: storage for every home:: storage ... causing an unnecessary query for every home storage initialized per request

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 6, 2017

Member

@butonic caching non-existing storages also sound good, could use the same cache keys ?

Member

PVince81 commented Mar 6, 2017

@butonic caching non-existing storages also sound good, could use the same cache keys ?

*/
public static function getStorageById($storageId) {
+ if (self::$localCache === null) {
+ self::$localCache = new CappedMemoryCache();

This comment has been minimized.

@PVince81

PVince81 Mar 6, 2017

Member

do we even need this one when memcache is there ?

@PVince81

PVince81 Mar 6, 2017

Member

do we even need this one when memcache is there ?

This comment has been minimized.

@butonic

butonic Mar 6, 2017

Member

saves trips to the distributet cache ... the availability wrapper constantly causes this method to be called. a local cache makes sense IMO.

@butonic

butonic Mar 6, 2017

Member

saves trips to the distributet cache ... the availability wrapper constantly causes this method to be called. a local cache makes sense IMO.

@@ -42,6 +44,14 @@ class Storage {
private $storageId;
private $numericId;
+ /** @var CappedMemoryCache */
+ protected static $localCache = null;

This comment has been minimized.

@DeepDiver1975

DeepDiver1975 Mar 6, 2017

Member

static? do we expect the cache to operate across instances? sounds strange ...

@DeepDiver1975

DeepDiver1975 Mar 6, 2017

Member

static? do we expect the cache to operate across instances? sounds strange ...

This comment has been minimized.

@DeepDiver1975

DeepDiver1975 Mar 6, 2017

Member

oh - getStorageById is static as well ...

@DeepDiver1975

DeepDiver1975 Mar 6, 2017

Member

oh - getStorageById is static as well ...

This comment has been minimized.

@butonic

butonic Mar 6, 2017

Member

yeah ... meh ...

@butonic

butonic Mar 6, 2017

Member

yeah ... meh ...

@DeepDiver1975

This comment has been minimized.

Show comment
Hide comment
@DeepDiver1975

DeepDiver1975 Mar 6, 2017

Member

There is some installation issue related to this change ....

11:29:48 Installing ....
11:29:48 ownCloud is not installed - only a limited number of commands are available
11:29:48 creating sqlite db
11:29:49 
11:29:49                                                                        
11:29:49   [RuntimeException]                                                   
11:29:49   Storage could neither be inserted nor be selected from the database  
11:29:49                                                                        
11:29:49 
11:29:49 Exception trace:
11:29:49  () at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Storage.php:79
11:29:49  OC\Files\Cache\Storage->__construct() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Common.php:377
11:29:49  OC\Files\Storage\Common->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
11:29:49  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
11:29:49  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Propagator.php:62
11:29:49  OC\Files\Cache\Propagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/HomePropagator.php:48
11:29:49  OC\Files\Cache\HomePropagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Updater.php:138
Member

DeepDiver1975 commented Mar 6, 2017

There is some installation issue related to this change ....

11:29:48 Installing ....
11:29:48 ownCloud is not installed - only a limited number of commands are available
11:29:48 creating sqlite db
11:29:49 
11:29:49                                                                        
11:29:49   [RuntimeException]                                                   
11:29:49   Storage could neither be inserted nor be selected from the database  
11:29:49                                                                        
11:29:49 
11:29:49 Exception trace:
11:29:49  () at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Storage.php:79
11:29:49  OC\Files\Cache\Storage->__construct() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Common.php:377
11:29:49  OC\Files\Storage\Common->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
11:29:49  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
11:29:49  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Propagator.php:62
11:29:49  OC\Files\Cache\Propagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/HomePropagator.php:48
11:29:49  OC\Files\Cache\HomePropagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Updater.php:138
@DeepDiver1975

This comment has been minimized.

Show comment
Hide comment
@DeepDiver1975

DeepDiver1975 Mar 6, 2017

Member

@butonic still failing to install ....

Member

DeepDiver1975 commented Mar 6, 2017

@butonic still failing to install ....

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 6, 2017

Member

@DeepDiver1975 this time it wasn't me:


12:18:09 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (21.735 secs / 20.435 secs)
12:18:11 �[33m06 03 2017 12:18:11.133:WARN [launcher]: �[39mPhantomJS was not killed in 2000 ms, sending SIGKILL.
12:18:11 Makefile:185: recipe for target 'test-js' failed
12:18:11 make: *** [test-js] Error 1
Member

butonic commented Mar 6, 2017

@DeepDiver1975 this time it wasn't me:


12:18:09 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (21.735 secs / 20.435 secs)
12:18:11 �[33m06 03 2017 12:18:11.133:WARN [launcher]: �[39mPhantomJS was not killed in 2000 ms, sending SIGKILL.
12:18:11 Makefile:185: recipe for target 'test-js' failed
12:18:11 make: *** [test-js] Error 1
@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 6, 2017

Member

the karma update strikes again...

Member

PVince81 commented Mar 6, 2017

the karma update strikes again...

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 6, 2017

Member

@PVince81 rebase and it's gone?

Member

butonic commented Mar 6, 2017

@PVince81 rebase and it's gone?

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 6, 2017

Member

Yes, rebase to try your luck again.

I have another PR where I setup a trap in hope to catch that bug if it appears again.

Member

PVince81 commented Mar 6, 2017

Yes, rebase to try your luck again.

I have another PR where I setup a trap in hope to catch that bug if it appears again.

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 6, 2017

Member
13:59:45 Using database oc_autotest20
13:59:45 Setup environment for sqlite testing on local storage ...
13:59:45 Installing ....
13:59:45 ownCloud is not installed - only a limited number of commands are available
13:59:45 creating sqlite db
13:59:46 
13:59:46                                                                        
13:59:46   [RuntimeException]                                                   
13:59:46   Storage could neither be inserted nor be selected from the database  
13:59:46                                                                        
13:59:46 
13:59:46 Exception trace:
13:59:46  () at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Storage.php:80
13:59:46  OC\Files\Cache\Storage->__construct() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Common.php:377
13:59:46  OC\Files\Storage\Common->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
13:59:46  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
13:59:46  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Propagator.php:62
13:59:46  OC\Files\Cache\Propagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/HomePropagator.php:48
13:59:46  OC\Files\Cache\HomePropagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Updater.php:138
13:59:46  OC\Files\Cache\Updater->update() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/View.php:313
13:59:46  OC\Files\View->writeUpdate() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/View.php:1137
13:59:46  OC\Files\View->basicOperation() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/View.php:261
13:59:46  OC\Files\View->mkdir() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Node/Folder.php:156
13:59:46  OC\Files\Node\Folder->newFolder() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Node/Root.php:355
13:59:46  OC\Files\Node\Root->getUserFolder() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Server.php:918
13:59:46  OC\Server->getUserFolder() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/User/Session.php:391
13:59:46  OC\User\Session->prepareUserLogin() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/User/Session.php:453
13:59:46  OC\User\Session->loginWithPassword() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/User/Session.php:290
13:59:46  OC\User\Session->login() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Setup.php:394
13:59:46  OC\Setup->install() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/core/Command/Maintenance/Install.php:86
13:59:46  OC\Core\Command\Maintenance\Install->execute() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Command/Command.php:262
13:59:46  Symfony\Component\Console\Command\Command->run() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Application.php:826
13:59:46  Symfony\Component\Console\Application->doRunCommand() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Application.php:189
13:59:46  Symfony\Component\Console\Application->doRun() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Application.php:120
13:59:46  Symfony\Component\Console\Application->run() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Console/Application.php:158
13:59:46  OC\Console\Application->run() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/console.php:99
13:59:46  require_once() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/occ:11
13:59:46 
13:59:46 maintenance:install [--database DATABASE] [--database-name DATABASE-NAME] [--database-host DATABASE-HOST] [--database-user DATABASE-USER] [--database-pass [DATABASE-PASS]] [--database-table-prefix [DATABASE-TABLE-PREFIX]] [--admin-user ADMIN-USER] [--admin-pass ADMIN-PASS] [--data-dir DATA-DIR]
13:59:46 
13:59:46 Makefile:177: recipe for target 'test-php' failed
13:59:46 make: *** [test-php] Error 1
Member

PVince81 commented Mar 6, 2017

13:59:45 Using database oc_autotest20
13:59:45 Setup environment for sqlite testing on local storage ...
13:59:45 Installing ....
13:59:45 ownCloud is not installed - only a limited number of commands are available
13:59:45 creating sqlite db
13:59:46 
13:59:46                                                                        
13:59:46   [RuntimeException]                                                   
13:59:46   Storage could neither be inserted nor be selected from the database  
13:59:46                                                                        
13:59:46 
13:59:46 Exception trace:
13:59:46  () at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Storage.php:80
13:59:46  OC\Files\Cache\Storage->__construct() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Common.php:377
13:59:46  OC\Files\Storage\Common->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
13:59:46  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Storage/Wrapper/Wrapper.php:448
13:59:46  OC\Files\Storage\Wrapper\Wrapper->getStorageCache() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Propagator.php:62
13:59:46  OC\Files\Cache\Propagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/HomePropagator.php:48
13:59:46  OC\Files\Cache\HomePropagator->propagateChange() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Cache/Updater.php:138
13:59:46  OC\Files\Cache\Updater->update() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/View.php:313
13:59:46  OC\Files\View->writeUpdate() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/View.php:1137
13:59:46  OC\Files\View->basicOperation() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/View.php:261
13:59:46  OC\Files\View->mkdir() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Node/Folder.php:156
13:59:46  OC\Files\Node\Folder->newFolder() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Files/Node/Root.php:355
13:59:46  OC\Files\Node\Root->getUserFolder() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Server.php:918
13:59:46  OC\Server->getUserFolder() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/User/Session.php:391
13:59:46  OC\User\Session->prepareUserLogin() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/User/Session.php:453
13:59:46  OC\User\Session->loginWithPassword() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/User/Session.php:290
13:59:46  OC\User\Session->login() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Setup.php:394
13:59:46  OC\Setup->install() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/core/Command/Maintenance/Install.php:86
13:59:46  OC\Core\Command\Maintenance\Install->execute() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Command/Command.php:262
13:59:46  Symfony\Component\Console\Command\Command->run() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Application.php:826
13:59:46  Symfony\Component\Console\Application->doRunCommand() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Application.php:189
13:59:46  Symfony\Component\Console\Application->doRun() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/composer/symfony/console/Application.php:120
13:59:46  Symfony\Component\Console\Application->run() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/lib/private/Console/Application.php:158
13:59:46  OC\Console\Application->run() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/console.php:99
13:59:46  require_once() at /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/occ:11
13:59:46 
13:59:46 maintenance:install [--database DATABASE] [--database-name DATABASE-NAME] [--database-host DATABASE-HOST] [--database-user DATABASE-USER] [--database-pass [DATABASE-PASS]] [--database-table-prefix [DATABASE-TABLE-PREFIX]] [--admin-user ADMIN-USER] [--admin-pass ADMIN-PASS] [--data-dir DATA-DIR]
13:59:46 
13:59:46 Makefile:177: recipe for target 'test-php' failed
13:59:46 make: *** [test-php] Error 1
@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 6, 2017

Member

@PVince81 @DeepDiver1975 hmmm do I just amend and force push until it passes the js stuff?

15:50:59 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (21.728 secs / 20.494 secs)
15:51:01 �[33m06 03 2017 15:51:01.775:WARN [launcher]: �[39mPhantomJS was not killed in 2000 ms, sending SIGKILL.
15:51:01 Makefile:185: recipe for target 'test-js' failed
15:51:01 make: *** [test-js] Error 1
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
15:51:01 Test execution failed: hudson.AbortException: script returned exit code 2
[Pipeline] step
15:51:01 Recording test results
Member

butonic commented Mar 6, 2017

@PVince81 @DeepDiver1975 hmmm do I just amend and force push until it passes the js stuff?

15:50:59 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (21.728 secs / 20.494 secs)
15:51:01 �[33m06 03 2017 15:51:01.775:WARN [launcher]: �[39mPhantomJS was not killed in 2000 ms, sending SIGKILL.
15:51:01 Makefile:185: recipe for target 'test-js' failed
15:51:01 make: *** [test-js] Error 1
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
15:51:01 Test execution failed: hudson.AbortException: script returned exit code 2
[Pipeline] step
15:51:01 Recording test results
@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 6, 2017

Member

or wait for #27196 to finish and be merged, then rebase.

It contains a potential killer for that bug.

Member

PVince81 commented Mar 6, 2017

or wait for #27196 to finish and be merged, then rebase.

It contains a potential killer for that bug.

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 6, 2017

Member

@butonic you can rebase now, the fix for the JS failure is on master

Member

PVince81 commented Mar 6, 2017

@butonic you can rebase now, the fix for the JS failure is on master

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 6, 2017

Member

@PVince81 pfffff. I rebased ... still does not like me :(

20:55:25 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) SUCCESS (24.379 secs / 23.186 secs)
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0) ERROR
20:55:25   TypeError: undefined is not a function (evaluating 'this.data.state()')
20:55:25   at apps/files/js/file-upload.js:166
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (24.602 secs / 23.186 secs)
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0) ERROR
20:55:25   TypeError: undefined is not a function (evaluating 'this.data.state()')
20:55:25   at apps/files/js/file-upload.js:166
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (24.605 secs / 23.186 secs)
20:55:27 �[33m06 03 2017 20:55:27.324:WARN [launcher]: �[39mPhantomJS was not killed in 2000 ms, sending SIGKILL.
20:55:27 Makefile:185: recipe for target 'test-js' failed
20:55:27 make: *** [test-js] Error 1
Member

butonic commented Mar 6, 2017

@PVince81 pfffff. I rebased ... still does not like me :(

20:55:25 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) SUCCESS (24.379 secs / 23.186 secs)
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0) ERROR
20:55:25   TypeError: undefined is not a function (evaluating 'this.data.state()')
20:55:25   at apps/files/js/file-upload.js:166
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (24.602 secs / 23.186 secs)
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0) ERROR
20:55:25   TypeError: undefined is not a function (evaluating 'this.data.state()')
20:55:25   at apps/files/js/file-upload.js:166
20:55:25 PhantomJS 2.1.1 (Linux 0.0.0): Executed 835 of 838 (skipped 3) ERROR (24.605 secs / 23.186 secs)
20:55:27 �[33m06 03 2017 20:55:27.324:WARN [launcher]: �[39mPhantomJS was not killed in 2000 ms, sending SIGKILL.
20:55:27 Makefile:185: recipe for target 'test-js' failed
20:55:27 make: *** [test-js] Error 1
@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 7, 2017

Member

I cannot stress enough how grateful I am that we added so many unit tests over the years.

22:58:16 There were 3 failures:
22:58:16 
22:58:16 1) OCA\Files_Sharing\Tests\BackendTest::testGetParents
22:58:16 Failed asserting that 0 is identical to 2.
22:58:16 
22:58:16 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/apps/files_sharing/tests/BackendTest.php:84
22:58:16 
22:58:16 2) OCA\Files_Sharing\Tests\SharedStorageTest::testFopenWithCreateOnlyPermission
22:58:16 Failed asserting that false is true.
22:58:16 
22:58:16 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/apps/files_sharing/tests/SharedStorageTest.php:303
22:58:16 
22:58:16 3) OCA\Files_Sharing\Tests\ApiTest::testDefaultExpireDate
22:58:16 Failed asserting that 0 is identical to 1.
22:58:16 
22:58:16 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/apps/files_sharing/tests/ApiTest.php:1422
Member

butonic commented Mar 7, 2017

I cannot stress enough how grateful I am that we added so many unit tests over the years.

22:58:16 There were 3 failures:
22:58:16 
22:58:16 1) OCA\Files_Sharing\Tests\BackendTest::testGetParents
22:58:16 Failed asserting that 0 is identical to 2.
22:58:16 
22:58:16 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/apps/files_sharing/tests/BackendTest.php:84
22:58:16 
22:58:16 2) OCA\Files_Sharing\Tests\SharedStorageTest::testFopenWithCreateOnlyPermission
22:58:16 Failed asserting that false is true.
22:58:16 
22:58:16 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/apps/files_sharing/tests/SharedStorageTest.php:303
22:58:16 
22:58:16 3) OCA\Files_Sharing\Tests\ApiTest::testDefaultExpireDate
22:58:16 Failed asserting that 0 is identical to 1.
22:58:16 
22:58:16 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/apps/files_sharing/tests/ApiTest.php:1422
@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 7, 2017

Member

BackendTest extends the Sharing TestCase. during setUp loginHelper is called, which calls resetStorage ... which does reflection magic to cause a reinitialization.

testFopenWithCreateOnlyPermission calls loginHelper, which calls resetStorage ...

testDefaultExpireDate calls loginHelper, which calls resetStorage ...

Investigating.

Member

butonic commented Mar 7, 2017

BackendTest extends the Sharing TestCase. during setUp loginHelper is called, which calls resetStorage ... which does reflection magic to cause a reinitialization.

testFopenWithCreateOnlyPermission calls loginHelper, which calls resetStorage ...

testDefaultExpireDate calls loginHelper, which calls resetStorage ...

Investigating.

butonic added some commits Mar 7, 2017

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 7, 2017

Member

tests are green

Member

butonic commented Mar 7, 2017

tests are green

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 7, 2017

Member

or were until i removed an unneeded use statement

Member

butonic commented Mar 7, 2017

or were until i removed an unneeded use statement

+ self::$localCache = new CappedMemoryCache();
+ }
+ $result = self::$localCache->get($storageId);
+ if ($result === null) {

This comment has been minimized.

@PVince81

PVince81 Mar 8, 2017

Member

should we distinguish between null meaning "no cache key, so try the other one" and null meaning "we know that this storage id does not exist" ? it could save a trip to the distributed cache if the local cache already knows about a known non-existence of a storage id.

@PVince81

PVince81 Mar 8, 2017

Member

should we distinguish between null meaning "no cache key, so try the other one" and null meaning "we know that this storage id does not exist" ? it could save a trip to the distributed cache if the local cache already knows about a known non-existence of a storage id.

This comment has been minimized.

@PVince81

PVince81 Mar 8, 2017

Member

never mind, I re-read your commits and saw that false is the value for non-existence which is subtly returned by the DB query.

@PVince81

PVince81 Mar 8, 2017

Member

never mind, I re-read your commits and saw that false is the value for non-existence which is subtly returned by the DB query.

- $result = \OC_DB::executeAudited($sql, [$storageId]);
- return $result->fetchRow();
+ $resultSet = \OC_DB::executeAudited($sql, [$storageId]);
+ return $resultSet->fetchRow();

This comment has been minimized.

@PVince81

PVince81 Mar 8, 2017

Member

ok got it, so this would return false and not null if the entry doesn't exist

@PVince81

PVince81 Mar 8, 2017

Member

ok got it, so this would return false and not null if the entry doesn't exist

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 8, 2017

Member

Code looks good. Let me try and find ways to break this...

Member

PVince81 commented Mar 8, 2017

Code looks good. Let me try and find ways to break this...

lib/private/Files/Cache/Storage.php
@@ -60,6 +70,7 @@ public function __construct($storage, $isAvailable = true) {
} else {
$connection = \OC::$server->getDatabaseConnection();
$available = $isAvailable ? 1 : 0;
+ self::unsetCache($this->storageId);
if ($connection->insertIfNotExist('*PREFIX*storages', ['id' => $this->storageId, 'available' => $available])) {
$this->numericId = (int)$connection->lastInsertId('*PREFIX*storages');

This comment has been minimized.

@PVince81

PVince81 Mar 8, 2017

Member

after inserting, store directly into cache ?

@PVince81

PVince81 Mar 8, 2017

Member

after inserting, store directly into cache ?

This comment has been minimized.

@butonic

butonic Mar 8, 2017

Member

yes, makes sense

@butonic

butonic Mar 8, 2017

Member

yes, makes sense

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 8, 2017

Member

Here we are assuming that all the code is always going through this class to manage storages.

In some cases like repair steps or background jobs (to be verified), storages might be deleted directly on the database in which case the distributed cache might still contain the old value when queried. I'm not sure whether there is a real-life situation where this could happen and be a bad thing because usually the deletion of storages through repair/bkg job are related to deleted users, so there shouldn't be any other process requesting it anyway.

Member

PVince81 commented Mar 8, 2017

Here we are assuming that all the code is always going through this class to manage storages.

In some cases like repair steps or background jobs (to be verified), storages might be deleted directly on the database in which case the distributed cache might still contain the old value when queried. I'm not sure whether there is a real-life situation where this could happen and be a bad thing because usually the deletion of storages through repair/bkg job are related to deleted users, so there shouldn't be any other process requesting it anyway.

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 8, 2017

Member

Only found one occurrence:

apps/files_sharing/lib/Command/CleanupRemoteStorages.php|110| $queryBuilder->delete('storages')

This is a situation where the remote storage is already orphaned so it's ok to delete it that way.
Since there is no oc_external_share entry anyway there is no code that could want to request it even if it's in the distributed cache.

I did a quick test with the following and all worked fine with and without redis:

  • federated shares
  • encryption
  • external storage + share originating from it (storageception)
  • run cron.php

So I only have two points:

  1. #27312 (comment)
  2. Here it's the clear() method, usually only used by tests. Do we also want to clear the caches just in case ?
lib/private/Files/Cache/Cache.php|549| $sql = 'DELETE FROM `*PREFIX*storages` WHERE `id` = ?';

Note to @owncloud/qa : we also need to run integration tests with redis / distributed cache enabled...

Member

PVince81 commented Mar 8, 2017

Only found one occurrence:

apps/files_sharing/lib/Command/CleanupRemoteStorages.php|110| $queryBuilder->delete('storages')

This is a situation where the remote storage is already orphaned so it's ok to delete it that way.
Since there is no oc_external_share entry anyway there is no code that could want to request it even if it's in the distributed cache.

I did a quick test with the following and all worked fine with and without redis:

  • federated shares
  • encryption
  • external storage + share originating from it (storageception)
  • run cron.php

So I only have two points:

  1. #27312 (comment)
  2. Here it's the clear() method, usually only used by tests. Do we also want to clear the caches just in case ?
lib/private/Files/Cache/Cache.php|549| $sql = 'DELETE FROM `*PREFIX*storages` WHERE `id` = ?';

Note to @owncloud/qa : we also need to run integration tests with redis / distributed cache enabled...

+ * @param string $storageId
+ * @return array|false
+ */
+ private static function getStorageByIdFromDb($storageId) {
$sql = 'SELECT * FROM `*PREFIX*storages` WHERE `id` = ?';

This comment has been minimized.

@mrow4a

mrow4a Mar 8, 2017

Contributor

Hmm, I remember @DeepDiver1975 telling me not to use hand-written queries

@mrow4a

mrow4a Mar 8, 2017

Contributor

Hmm, I remember @DeepDiver1975 telling me not to use hand-written queries

This comment has been minimized.

@butonic

butonic Mar 8, 2017

Member

was already there ... improving existing sql is not part of this PR

@butonic

butonic Mar 8, 2017

Member

was already there ... improving existing sql is not part of this PR

This comment has been minimized.

@butonic

butonic Mar 8, 2017

Member

then we could have a look at why all this crap is still static ... also not part of this PR.

@butonic

butonic Mar 8, 2017

Member

then we could have a look at why all this crap is still static ... also not part of this PR.

lib/private/Files/Cache/Storage.php
+ // distributed cache may need initializing
+ if (self::$distributedCache === null) {
+ self::$distributedCache =
+ \OC::$server->getMemCacheFactory()->create('getStorageById');

This comment has been minimized.

@PVince81

PVince81 Mar 9, 2017

Member

self::$cacheKey ? I'm seeing a lot of duplicated code for cache initialization now.
I was hoping this block here would be a one-liner "setTheDamnThingInWhateverCache()"

@PVince81

PVince81 Mar 9, 2017

Member

self::$cacheKey ? I'm seeing a lot of duplicated code for cache initialization now.
I was hoping this block here would be a one-liner "setTheDamnThingInWhateverCache()"

This comment has been minimized.

@butonic

butonic Mar 9, 2017

Member

now there is only one place where getStorageById is used. No need for another const I think.

@butonic

butonic Mar 9, 2017

Member

now there is only one place where getStorageById is used. No need for another const I think.

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 9, 2017

Member

Great, the code looks better now!

Now hopefully the last thing:

13:01:54 1) Test\Files\External\Service\UserStoragesServiceTest::testDeleteStorage with data set #1 (array('share', 'example.com', '$user', 'testPassword', 'someroot'), 'smb::someone@example.com/shar...meroot', 1)
13:01:54 Failed asserting that actual size 0 matches expected size 1.
13:01:54 
13:01:54 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/tests/lib/Files/External/Service/StoragesServiceTest.php:304
13:01:54 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/tests/lib/Files/External/Service/UserStoragesServiceTest.php:150
13:01:54 
Member

PVince81 commented Mar 9, 2017

Great, the code looks better now!

Now hopefully the last thing:

13:01:54 1) Test\Files\External\Service\UserStoragesServiceTest::testDeleteStorage with data set #1 (array('share', 'example.com', '$user', 'testPassword', 'someroot'), 'smb::someone@example.com/shar...meroot', 1)
13:01:54 Failed asserting that actual size 0 matches expected size 1.
13:01:54 
13:01:54 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/tests/lib/Files/External/Service/StoragesServiceTest.php:304
13:01:54 /var/lib/jenkins/workspace/owncloud-core_core_PR-27312-IS2RVGGWD5ND2C634M4Y7UPQE2VO4H4YZCTWBRGZ7VCAC653ZGJA/tests/lib/Files/External/Service/UserStoragesServiceTest.php:150
13:01:54 

@PVince81 PVince81 merged commit d113631 into master Mar 9, 2017

4 checks passed

Scrutinizer 4 new issues, 4 updated code elements
Details
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
licence/cla Contributor License Agreement is signed.
Details

@PVince81 PVince81 deleted the cache-getStorageById branch Mar 9, 2017

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 9, 2017

Member

Congrats! backport ?

Member

PVince81 commented Mar 9, 2017

Congrats! backport ?

@butonic

This comment has been minimized.

Show comment
Hide comment
@butonic

butonic Mar 9, 2017

Member

on it

Member

butonic commented Mar 9, 2017

on it

@PVince81

This comment has been minimized.

Show comment
Hide comment
@PVince81

PVince81 Mar 10, 2017

Member

stable9.1: #27349
stable9: #27350

Member

PVince81 commented Mar 10, 2017

stable9.1: #27349
stable9: #27350

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment