Skip to content

Commit

Permalink
Split advisory rules files in many to manage variables removed in som…
Browse files Browse the repository at this point in the history 
…e versions

Signed-off-by: Hugues Peccatte <hugues.peccatte@gmail.com>
  • Loading branch information
@Tithugues
Tithugues committed Aug 30, 2019
1 parent ab8ffbc commit 99b2dfa
Show file tree
Hide file tree
Showing 3 changed files with 85 additions and 62 deletions.
50 changes: 0 additions & 50 deletions libraries/advisory_rules.txt → libraries/advisory_rules_generic.txt
View file
Expand Up @@ -142,63 +142,13 @@ rule 'MySQL Architecture'
#
# Query cache

# Lame: 'ON' == 0 is true, so you need to compare 'ON' == '0'
rule 'Query cache disabled' [IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003]
query_cache_size
value == 0 || query_cache_type == 'OFF' || query_cache_type == '0'
The query cache is not enabled.
The query cache is known to greatly improve performance if configured correctly. Enable it by setting {query_cache_size} to a 2 digit MiB value and setting {query_cache_type} to 'ON'. <b>Note:</b> If you are using memcached, ignore this recommendation.
query_cache_size is set to 0 or query_cache_type is set to 'OFF'

rule 'Query caching method' [!fired('Query cache disabled')]
Questions / Uptime
value > 100
Suboptimal caching method.
You are using the MySQL Query cache with a fairly high traffic database. It might be worth considering to use <a href="https://dev.mysql.com/doc/refman/5.5/en/ha-memcached.html">memcached</a> instead of the MySQL Query cache, especially if you have multiple slaves.
The query cache is enabled and the server receives %d queries per second. This rule fires if there is more than 100 queries per second. | round(value,1)

rule 'Query cache efficiency (%)' [(IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003) && Com_select + Qcache_hits > 0 && !fired('Query cache disabled')]
Qcache_hits / (Com_select + Qcache_hits) * 100
value < 20
Query cache not running efficiently, it has a low hit rate.
Consider increasing {query_cache_limit}.
The current query cache hit rate of %s% is below 20% | round(value,1)

rule 'Query Cache usage' [(IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003) && !fired('Query cache disabled')]
100 - Qcache_free_memory / query_cache_size * 100
value < 80
Less than 80% of the query cache is being utilized.
This might be caused by {query_cache_limit} being too low. Flushing the query cache might help as well.
The current ratio of free query cache memory to total query cache size is %s%. It should be above 80% | round(value,1)

rule 'Query cache fragmentation' [(IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003) && !fired('Query cache disabled')]
Qcache_free_blocks / (Qcache_total_blocks / 2) * 100
value > 20
The query cache is considerably fragmented.
Severe fragmentation is likely to (further) increase Qcache_lowmem_prunes. This might be caused by many Query cache low memory prunes due to {query_cache_size} being too small. For a immediate but short lived fix you can flush the query cache (might lock the query cache for a long time). Carefully adjusting {query_cache_min_res_unit} to a lower value might help too, e.g. you can set it to the average size of your queries in the cache using this formula: (query_cache_size - qcache_free_memory) / qcache_queries_in_cache
The cache is currently fragmented by %s% , with 100% fragmentation meaning that the query cache is an alternating pattern of free and used blocks. This value should be below 20%. | round(value,1)

rule 'Query cache low memory prunes' [(IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003) && Qcache_inserts > 0 && !fired('Query cache disabled')]
Qcache_lowmem_prunes / Qcache_inserts * 100
value > 0.1
Cached queries are removed due to low query cache memory from the query cache.
You might want to increase {query_cache_size}, however keep in mind that the overhead of maintaining the cache is likely to increase with its size, so do this in small increments and monitor the results.
The ratio of removed queries to inserted queries is %s%. The lower this value is, the better (This rules firing limit: 0.1%) | round(value,1)

rule 'Query cache max size' [(IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003) && !fired('Query cache disabled')]
query_cache_size
value > 1024 * 1024 * 128
The query cache size is above 128 MiB. Big query caches may cause significant overhead that is required to maintain the cache.
Depending on your environment, it might be performance increasing to reduce this value.
Current query cache size: %s | ADVISOR_formatByteDown(value, 2, 2)

rule 'Query cache min result size' [(IS_MYSQL == false || PMA_MYSQL_INT_VERSION < 80003) && !fired('Query cache disabled')]
query_cache_limit
value == 1024*1024
The max size of the result set in the query cache is the default of 1 MiB.
Changing {query_cache_limit} (usually by increasing) may increase efficiency. This variable determines the maximum size a query result may have to be inserted into the query cache. If there are many query results above 1 MiB that are well cacheable (many reads, little writes) then increasing {query_cache_limit} will increase efficiency. Whereas in the case of many query results being above 1 MiB that are not very well cacheable (often invalidated due to table updates) increasing {query_cache_limit} might reduce efficiency.
query_cache_limit is set to 1 MiB

#
# Sorts
rule 'Percentage of sorts that cause temporary tables' [Sort_scan + Sort_range > 0]
Expand Down
57 changes: 57 additions & 0 deletions libraries/advisory_rules_mysql_before80003.txt
View file
@@ -0,0 +1,57 @@
# phpMyAdmin Advisory rules file
#
# See doc in advisory_rules_generic.txt
#

#
# Query cache

# Lame: 'ON' == 0 is true, so you need to compare 'ON' == '0'
rule 'Query cache disabled'
query_cache_size
value == 0 || query_cache_type == 'OFF' || query_cache_type == '0'
The query cache is not enabled.
The query cache is known to greatly improve performance if configured correctly. Enable it by setting {query_cache_size} to a 2 digit MiB value and setting {query_cache_type} to 'ON'. <b>Note:</b> If you are using memcached, ignore this recommendation.
query_cache_size is set to 0 or query_cache_type is set to 'OFF'

rule 'Query cache efficiency (%)' [Com_select + Qcache_hits > 0 && !fired('Query cache disabled')]
Qcache_hits / (Com_select + Qcache_hits) * 100
value < 20
Query cache not running efficiently, it has a low hit rate.
Consider increasing {query_cache_limit}.
The current query cache hit rate of %s% is below 20% | round(value,1)

rule 'Query Cache usage' [!fired('Query cache disabled')]
100 - Qcache_free_memory / query_cache_size * 100
value < 80
Less than 80% of the query cache is being utilized.
This might be caused by {query_cache_limit} being too low. Flushing the query cache might help as well.
The current ratio of free query cache memory to total query cache size is %s%. It should be above 80% | round(value,1)

rule 'Query cache fragmentation' [!fired('Query cache disabled')]
Qcache_free_blocks / (Qcache_total_blocks / 2) * 100
value > 20
The query cache is considerably fragmented.
Severe fragmentation is likely to (further) increase Qcache_lowmem_prunes. This might be caused by many Query cache low memory prunes due to {query_cache_size} being too small. For a immediate but short lived fix you can flush the query cache (might lock the query cache for a long time). Carefully adjusting {query_cache_min_res_unit} to a lower value might help too, e.g. you can set it to the average size of your queries in the cache using this formula: (query_cache_size - qcache_free_memory) / qcache_queries_in_cache
The cache is currently fragmented by %s% , with 100% fragmentation meaning that the query cache is an alternating pattern of free and used blocks. This value should be below 20%. | round(value,1)

rule 'Query cache low memory prunes' [Qcache_inserts > 0 && !fired('Query cache disabled')]
Qcache_lowmem_prunes / Qcache_inserts * 100
value > 0.1
Cached queries are removed due to low query cache memory from the query cache.
You might want to increase {query_cache_size}, however keep in mind that the overhead of maintaining the cache is likely to increase with its size, so do this in small increments and monitor the results.
The ratio of removed queries to inserted queries is %s%. The lower this value is, the better (This rules firing limit: 0.1%) | round(value,1)

rule 'Query cache max size' [!fired('Query cache disabled')]
query_cache_size
value > 1024 * 1024 * 128
The query cache size is above 128 MiB. Big query caches may cause significant overhead that is required to maintain the cache.
Depending on your environment, it might be performance increasing to reduce this value.
Current query cache size: %s | ADVISOR_formatByteDown(value, 2, 2)

rule 'Query cache min result size' [!fired('Query cache disabled')]
query_cache_limit
value == 1024*1024
The max size of the result set in the query cache is the default of 1 MiB.
Changing {query_cache_limit} (usually by increasing) may increase efficiency. This variable determines the maximum size a query result may have to be inserted into the query cache. If there are many query results above 1 MiB that are well cacheable (many reads, little writes) then increasing {query_cache_limit} will increase efficiency. Whereas in the case of many query results being above 1 MiB that are not very well cacheable (often invalidated due to table updates) increasing {query_cache_limit} might reduce efficiency.
query_cache_limit is set to 1 MiB
40 changes: 28 additions & 12 deletions libraries/classes/Advisor.php
View file
Expand Up @@ -221,12 +221,15 @@ public function run()
$this->variables['system_memory']
= isset($memory['MemTotal']) ? $memory['MemTotal'] : 0;

// Add IS_MYSQL to globals
$isMariaDB = false !== strpos($this->getVariables()['version'], 'MariaDB');
$this->globals['IS_MYSQL'] = !$isMariaDB;
$ruleFiles = $this->defineRulesFiles();

// Step 2: Read and parse the list of rules
$this->setParseResult(static::parseRulesFile());
$parsedResults = [];
foreach ($ruleFiles as $ruleFile) {
$parsedResults[] = $this->parseRulesFile($ruleFile);
}
$this->setParseResult(call_user_func_array('array_merge_recursive', $parsedResults));

// Step 3: Feed the variables to the rules and let them fire. Sets
// $runResult
$this->runRules();
Expand Down Expand Up @@ -433,6 +436,22 @@ public function addRule($type, array $rule)
$this->runResult[$type][] = $rule;
}

/**
* Defines the rules files to use
*
* @return array
*/
protected function defineRulesFiles()
{
$isMariaDB = false !== strpos($this->getVariables()['version'], 'MariaDB');
$ruleFiles = ['libraries/advisory_rules_generic.txt'];
// If MariaDB (= not MySQL) OR MYSQL < 8.0.3, add another rules file.
if ($isMariaDB || $this->globals['PMA_MYSQL_INT_VERSION'] < 80003) {
$ruleFiles[] = 'libraries/advisory_rules_mysql_before80003.txt';
}
return $ruleFiles;
}

/**
* Callback for wrapping links with Core::linkURL
*
Expand Down Expand Up @@ -484,23 +503,23 @@ public function ruleExprEvaluate($expr)
* Reads the rule file into an array, throwing errors messages on syntax
* errors.
*
* @param string $filename Name of file to parse
*
* @return array with parsed data
*/
public static function parseRulesFile()
public static function parseRulesFile($filename)
{
$filename = 'libraries/advisory_rules.txt';
$file = file($filename, FILE_IGNORE_NEW_LINES);

$errors = array();
$rules = array();
$lines = array();

if ($file === false) {
$errors[] = sprintf(
__('Error in reading file: The file \'%s\' does not exist or is not readable!'),
$filename
);
return array('rules' => $rules, 'lines' => $lines, 'errors' => $errors);
return array('rules' => $rules, 'errors' => $errors);
}

$ruleSyntax = array(
Expand Down Expand Up @@ -534,10 +553,8 @@ public static function parseRulesFile()
$ruleLine = 1;
$ruleNo++;
$rules[$ruleNo] = array('name' => $match[1]);
$lines[$ruleNo] = array('name' => $i + 1);
if (isset($match[3])) {
$rules[$ruleNo]['precondition'] = $match[3];
$lines[$ruleNo]['precondition'] = $i + 1;
}
} else {
$errors[] = sprintf(
Expand Down Expand Up @@ -575,7 +592,6 @@ public static function parseRulesFile()
$rules[$ruleNo][$ruleSyntax[$ruleLine]] = chop(
mb_substr($line, 1)
);
$lines[$ruleNo][$ruleSyntax[$ruleLine]] = $i + 1;
++$ruleLine;
}

Expand All @@ -585,7 +601,7 @@ public static function parseRulesFile()
}
}

return array('rules' => $rules, 'lines' => $lines, 'errors' => $errors);
return array('rules' => $rules, 'errors' => $errors);
}

/**
Expand Down

0 comments on commit 99b2dfa

Please sign in to comment.