Skip to content
This repository has been archived by the owner on Apr 4, 2024. It is now read-only.

use cache on dynamic group #2058

Merged
merged 3 commits into from May 10, 2017

Conversation

orthagh
Copy link
Contributor

@orthagh orthagh commented Feb 7, 2017

When demanding a b/collect/index.php?action=getJobs ,
All agent are computed again and so a dynamic group listing launch.
Depending on the criteria saved in this group, query could be very long and executed again for each agent of this group (imagine a deployment for 1000 computers querying the dynamic group each).

If i understand well, the code check if an agent always belongs to a prepared job and cancel if not.
For mass deployment, it's pretty aggressive on the server.

I suggest, in this pr, to cache computers_id of dynamic groups at the preparation of the task (use_cache = false, the default), and at the moment of execution, use this cache (use_cache = true).

Maybe, we can do better for this problem, but i don't see atm.

Also cleaned some incoherence in update.php

@orthagh orthagh added the bug label Feb 7, 2017
@orthagh orthagh requested a review from ddurieux February 7, 2017 16:52
@coveralls
Copy link

coveralls commented Feb 7, 2017

Coverage Status

Coverage increased (+0.07%) to 26.847% when pulling 4db14a0 on orthagh:cache_dynamic_groups into e8bb67f on fusioninventory:glpi9.1.

static function getTargetsByGroup(PluginFusioninventoryDeployGroup $group, $use_cache = false) {
$ids = array();

if (!$use_cache || !$ids = self::retrieveCache($group)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really agree with that, if you call the function with $use_cache = 'false' and you have ids in cache, it use cache.
So 2 options:

  • if we not use cache, really not use it
  • keep like it but remove the param $use_cache in the function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe the notation is a little too compact but here is the different cases:

  • $use_cache = false, the first part of the 'if' will trigger (!$use_cache) and the second part will not (due to the '|' usage)
  • $use_cache = true, the first part return false, so we try with the second part to load the cache:
    • it succeeds: we skip the block
    • it fails: we execute the block and load data from Search engine.

For resume, as the mentionned 'if' have an OR condition, if the first part is evaluated as true, the second part will not be evaluated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, the second part will not be called if $use_cache is set to false.

@@ -915,6 +915,7 @@ CREATE TABLE `glpi_plugin_fusioninventory_deploygroups_dynamicdatas` (
`plugin_fusioninventory_deploygroups_id` int(11) NOT NULL DEFAULT '0',
`fields_array` text DEFAULT NULL,
`can_update_group` tinyint(1) NOT NULL DEFAULT '0',
`computers_id_cache` text DEFAULT NULL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On big computers database, perhaps use longtext or blob

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but, should not blob cause issues (I'm thinking about future multi-database support mainly)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think about that too, but postgresql use bytea instead blob and the text is unlimited in size instead MySQL, so in all cases, will need modify it :p
I preferer longtext now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 for longtext

Copy link
Contributor

@trasher trasher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

static function getTargetsByGroup(PluginFusioninventoryDeployGroup $group, $use_cache = false) {
$ids = array();

if (!$use_cache || !$ids = self::retrieveCache($group)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, the second part will not be called if $use_cache is set to false.

return $ids;
}

static function storeCache(PluginFusioninventoryDeployGroup $group, $ids) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add phpdoc

return $DB->query($query);
}

static function retrieveCache(PluginFusioninventoryDeployGroup $group) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add phpdoc

@orthagh
Copy link
Contributor Author

orthagh commented May 3, 2017

Any news on this PR ?
Did you see my answer above ?

@ddurieux
Copy link
Member

Ok, I'm ok with that, you can merge ;)

@orthagh
Copy link
Contributor Author

orthagh commented May 10, 2017

Thank you !

@orthagh orthagh merged commit 0989feb into fusioninventory:glpi9.1 May 10, 2017
orthagh added a commit that referenced this pull request May 10, 2017
* use cache on dynamic group

* use longtext for cache

* add dockblock
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants