Extend property map with compaction stats #2794
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@Tema has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Summary: This branch extends existing property map which keeps values in doubles to keep values in strings so that it can be used to provide wider range of properties. The immediate need for that is to provide IO stall stats in an easy parseable way to MyRocks which is also part of this branch. Test Plan: Modify the existing tests which uses doubles to use strings. Added IO stall are tested thought MyRocks test.
|
@Tema updated the pull request - view changes - changes since last import |
db/internal_stats.cc
Outdated
| cf_stats_count_[LEVEL0_NUM_FILES_TOTAL] + | ||
| cf_stats_count_[SOFT_PENDING_COMPACTION_BYTES_LIMIT] + | ||
| cf_stats_count_[HARD_PENDING_COMPACTION_BYTES_LIMIT] + | ||
| cf_stats_count_[MEMTABLE_COMPACTION] + cf_stats_count_[MEMTABLE_SLOWDOWN]; |
There was a problem hiding this comment.
There are 2 kinds of stalls
slow down stalls -> Writes continue but with slow down penalty
stopping stalls -> Writes totally stop
I think we should have two different counters ("total_slowdown_stall", "total_stop_stall"
Also applied clang-format for modified code
|
@Tema updated the pull request - view changes - changes since last import |
|
@Tema has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
ajkr
reviewed
Aug 30, 2017
| uint64_t total_stop = | ||
| cf_stats_count_[LEVEL0_NUM_FILES_TOTAL] + | ||
| cf_stats_count_[HARD_PENDING_COMPACTION_BYTES_LIMIT] + | ||
| cf_stats_count_[MEMTABLE_COMPACTION]; |
There was a problem hiding this comment.
we should rename these constants (unrelated to your diff), they aren't descriptive at all.
ajkr
approved these changes
Aug 30, 2017
|
@Tema has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
ajkr
reviewed
Sep 1, 2017
| void InternalStats::DumpCFMapStatsIOStalls( | ||
| std::map<std::string, std::string>* cf_stats) { | ||
| (*cf_stats)["io_stalls.level0_slowdown"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_SLOWDOWN_TOTAL]); |
There was a problem hiding this comment.
L0_FILE_COUNT_LIMIT_SLOWDOWNS
| (*cf_stats)["io_stalls.level0_slowdown"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_SLOWDOWN_TOTAL]); | ||
| (*cf_stats)["io_stalls.level0_slowdown_with_compaction"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_SLOWDOWN_WITH_COMPACTION]); |
There was a problem hiding this comment.
LOCKED_L0_FILE_COUNT_LIMIT_SLOWDOWNS
| (*cf_stats)["io_stalls.level0_slowdown_with_compaction"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_SLOWDOWN_WITH_COMPACTION]); | ||
| (*cf_stats)["io_stalls.level0_numfiles"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_NUM_FILES_TOTAL]); |
| (*cf_stats)["io_stalls.level0_numfiles"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_NUM_FILES_TOTAL]); | ||
| (*cf_stats)["io_stalls.level0_numfiles_with_compaction"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_NUM_FILES_WITH_COMPACTION]); |
There was a problem hiding this comment.
LOCKED_L0_FILE_COUNT_LIMIT_STOPS
| (*cf_stats)["io_stalls.level0_numfiles_with_compaction"] = | ||
| std::to_string(cf_stats_count_[LEVEL0_NUM_FILES_WITH_COMPACTION]); | ||
| (*cf_stats)["io_stalls.stop_for_pending_compaction_bytes"] = | ||
| std::to_string(cf_stats_count_[HARD_PENDING_COMPACTION_BYTES_LIMIT]); |
There was a problem hiding this comment.
PENDING_COMPACTION_LIMIT_STOPS
| (*cf_stats)["io_stalls.stop_for_pending_compaction_bytes"] = | ||
| std::to_string(cf_stats_count_[HARD_PENDING_COMPACTION_BYTES_LIMIT]); | ||
| (*cf_stats)["io_stalls.slowdown_for_pending_compaction_bytes"] = | ||
| std::to_string(cf_stats_count_[SOFT_PENDING_COMPACTION_BYTES_LIMIT]); |
There was a problem hiding this comment.
PENDING_COMPACTION_LIMIT_SLOWDOWNS
| (*cf_stats)["io_stalls.slowdown_for_pending_compaction_bytes"] = | ||
| std::to_string(cf_stats_count_[SOFT_PENDING_COMPACTION_BYTES_LIMIT]); | ||
| (*cf_stats)["io_stalls.memtable_compaction"] = | ||
| std::to_string(cf_stats_count_[MEMTABLE_COMPACTION]); |
| (*cf_stats)["io_stalls.memtable_compaction"] = | ||
| std::to_string(cf_stats_count_[MEMTABLE_COMPACTION]); | ||
| (*cf_stats)["io_stalls.memtable_slowdown"] = | ||
| std::to_string(cf_stats_count_[MEMTABLE_SLOWDOWN]); |
facebook-github-bot
pushed a commit
to facebook/mysql-5.6
that referenced
this pull request
Sep 12, 2017
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes #695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Oct 26, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Oct 26, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Oct 27, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Oct 27, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Oct 30, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Oct 31, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
george-lorch
pushed a commit
to george-lorch/percona-server
that referenced
this pull request
Nov 1, 2017
Upstream commit ID : fb-mysql-5.6.35/3f9ff433589a3ab5cb94e249fa649835ee077321 Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 21b4c50
facebook-github-bot
pushed a commit
to facebook/mysql-5.6
that referenced
this pull request
Dec 23, 2019
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes #695 Test Plan: * Check that compaction level metrics are still published to INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS properly * Check the output of the command: ``` mysql> show status like 'rocksdb_stall_%'; +-----------------------------------------------------+-------+ | Variable_name | Value | +-----------------------------------------------------+-------+ | rocksdb_stall_level0_slowdown | 2 | | rocksdb_stall_level0_slowdown_with_compaction | 4 | | rocksdb_stall_level0_numfiles | 10 | | rocksdb_stall_level0_numfiles_with_compaction | 12 | | rocksdb_stall_stop_for_pending_compaction_bytes | 16 | | rocksdb_stall_slowdown_for_pending_compaction_bytes | 14 | | rocksdb_stall_memtable_compaction | 6 | | rocksdb_stall_memtable_slowdown | 8 | | rocksdb_stall_total_count | 56 | | rocksdb_stall_micros | 0 | +-----------------------------------------------------+-------+ ``` (In order to have some non-zero numbers, I've initialized internal RocksDB stall stats with 1-8 numbers. They all doubled in output as there are two column families in my instance: default and system. The stall_micros has been introduced previously. total_count doesn't include level0_slowdown_with_compaction and numfiles_with_compaction as only their total counterparts are included in this counter.) Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 1977338
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 12, 2020
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Test Plan: * Check that compaction level metrics are still published to INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS properly * Check the output of the command: ``` mysql> show status like 'rocksdb_stall_%'; +-----------------------------------------------------+-------+ | Variable_name | Value | +-----------------------------------------------------+-------+ | rocksdb_stall_level0_slowdown | 2 | | rocksdb_stall_level0_slowdown_with_compaction | 4 | | rocksdb_stall_level0_numfiles | 10 | | rocksdb_stall_level0_numfiles_with_compaction | 12 | | rocksdb_stall_stop_for_pending_compaction_bytes | 16 | | rocksdb_stall_slowdown_for_pending_compaction_bytes | 14 | | rocksdb_stall_memtable_compaction | 6 | | rocksdb_stall_memtable_slowdown | 8 | | rocksdb_stall_total_count | 56 | | rocksdb_stall_micros | 0 | +-----------------------------------------------------+-------+ ``` (In order to have some non-zero numbers, I've initialized internal RocksDB stall stats with 1-8 numbers. They all doubled in output as there are two column families in my instance: default and system. The stall_micros has been introduced previously. total_count doesn't include level0_slowdown_with_compaction and numfiles_with_compaction as only their total counterparts are included in this counter.) Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 1977338
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Sep 9, 2020
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Test Plan: * Check that compaction level metrics are still published to INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS properly * Check the output of the command: ``` mysql> show status like 'rocksdb_stall_%'; +-----------------------------------------------------+-------+ | Variable_name | Value | +-----------------------------------------------------+-------+ | rocksdb_stall_level0_slowdown | 2 | | rocksdb_stall_level0_slowdown_with_compaction | 4 | | rocksdb_stall_level0_numfiles | 10 | | rocksdb_stall_level0_numfiles_with_compaction | 12 | | rocksdb_stall_stop_for_pending_compaction_bytes | 16 | | rocksdb_stall_slowdown_for_pending_compaction_bytes | 14 | | rocksdb_stall_memtable_compaction | 6 | | rocksdb_stall_memtable_slowdown | 8 | | rocksdb_stall_total_count | 56 | | rocksdb_stall_micros | 0 | +-----------------------------------------------------+-------+ ``` (In order to have some non-zero numbers, I've initialized internal RocksDB stall stats with 1-8 numbers. They all doubled in output as there are two column families in my instance: default and system. The stall_micros has been introduced previously. total_count doesn't include level0_slowdown_with_compaction and numfiles_with_compaction as only their total counterparts are included in this counter.) Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 1977338
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Sep 16, 2020
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Test Plan: * Check that compaction level metrics are still published to INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS properly * Check the output of the command: ``` mysql> show status like 'rocksdb_stall_%'; +-----------------------------------------------------+-------+ | Variable_name | Value | +-----------------------------------------------------+-------+ | rocksdb_stall_level0_slowdown | 2 | | rocksdb_stall_level0_slowdown_with_compaction | 4 | | rocksdb_stall_level0_numfiles | 10 | | rocksdb_stall_level0_numfiles_with_compaction | 12 | | rocksdb_stall_stop_for_pending_compaction_bytes | 16 | | rocksdb_stall_slowdown_for_pending_compaction_bytes | 14 | | rocksdb_stall_memtable_compaction | 6 | | rocksdb_stall_memtable_slowdown | 8 | | rocksdb_stall_total_count | 56 | | rocksdb_stall_micros | 0 | +-----------------------------------------------------+-------+ ``` (In order to have some non-zero numbers, I've initialized internal RocksDB stall stats with 1-8 numbers. They all doubled in output as there are two column families in my instance: default and system. The stall_micros has been introduced previously. total_count doesn't include level0_slowdown_with_compaction and numfiles_with_compaction as only their total counterparts are included in this counter.) Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 1977338
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Oct 5, 2020
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Test Plan: * Check that compaction level metrics are still published to INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS properly * Check the output of the command: ``` mysql> show status like 'rocksdb_stall_%'; +-----------------------------------------------------+-------+ | Variable_name | Value | +-----------------------------------------------------+-------+ | rocksdb_stall_level0_slowdown | 2 | | rocksdb_stall_level0_slowdown_with_compaction | 4 | | rocksdb_stall_level0_numfiles | 10 | | rocksdb_stall_level0_numfiles_with_compaction | 12 | | rocksdb_stall_stop_for_pending_compaction_bytes | 16 | | rocksdb_stall_slowdown_for_pending_compaction_bytes | 14 | | rocksdb_stall_memtable_compaction | 6 | | rocksdb_stall_memtable_slowdown | 8 | | rocksdb_stall_total_count | 56 | | rocksdb_stall_micros | 0 | +-----------------------------------------------------+-------+ ``` (In order to have some non-zero numbers, I've initialized internal RocksDB stall stats with 1-8 numbers. They all doubled in output as there are two column families in my instance: default and system. The stall_micros has been introduced previously. total_count doesn't include level0_slowdown_with_compaction and numfiles_with_compaction as only their total counterparts are included in this counter.) Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 1977338
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Nov 11, 2020
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Test Plan: * Check that compaction level metrics are still published to INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS properly * Check the output of the command: ``` mysql> show status like 'rocksdb_stall_%'; +-----------------------------------------------------+-------+ | Variable_name | Value | +-----------------------------------------------------+-------+ | rocksdb_stall_level0_slowdown | 2 | | rocksdb_stall_level0_slowdown_with_compaction | 4 | | rocksdb_stall_level0_numfiles | 10 | | rocksdb_stall_level0_numfiles_with_compaction | 12 | | rocksdb_stall_stop_for_pending_compaction_bytes | 16 | | rocksdb_stall_slowdown_for_pending_compaction_bytes | 14 | | rocksdb_stall_memtable_compaction | 6 | | rocksdb_stall_memtable_slowdown | 8 | | rocksdb_stall_total_count | 56 | | rocksdb_stall_micros | 0 | +-----------------------------------------------------+-------+ ``` (In order to have some non-zero numbers, I've initialized internal RocksDB stall stats with 1-8 numbers. They all doubled in output as there are two column families in my instance: default and system. The stall_micros has been introduced previously. total_count doesn't include level0_slowdown_with_compaction and numfiles_with_compaction as only their total counterparts are included in this counter.) Differential Revision: D5723305 Pulled By: Tema fbshipit-source-id: 1977338
facebook-github-bot
pushed a commit
to facebook/mysql-5.6
that referenced
this pull request
Mar 8, 2021
Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes #695 Differential Revision: D5723305 (3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 16, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 17, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 30, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 30, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 31, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Sep 1, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Sep 2, 2021
…cebook#695) Summary: update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 (facebook@3f9ff43) Pulled By: Tema fbshipit-source-id: 688fb48f15f
hermanlee
pushed a commit
to facebook/mysql-5.6
that referenced
this pull request
Jan 10, 2022
Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes #695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jan 17, 2022
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
ldonoso
pushed a commit
to ldonoso/percona-server
that referenced
this pull request
Mar 15, 2022
…cona#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 20, 2022
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 23, 2022
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
laurynas-biveinis
pushed a commit
to laurynas-biveinis/mysql-5.6
that referenced
this pull request
Aug 11, 2022
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Mar 25, 2023
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Mar 28, 2023
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 1, 2023
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 14, 2023
…cebook#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook#695 Differential Revision: D5723305 Pulled By: Tema
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Apr 9, 2024
…cona#695) Summary: @update-submodule: rocksdb The stall counters are already shown in 'SHOW ENGINE ROCKSDB STATUS output' but this command is a bit expensive to run and you have to parse the output to monitor these values. This branch adds these counters to structured output of SHOW GLOBAL STATUS command. Also fixed the bug in INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS which showed only integer values but not float. The rocksdb module was updated to bring [this](facebook/rocksdb#2794 (review)) change which has the following changes to GetMapProperty method: 1. Changed values from double to string 2. Change existing keys to start with "compaction." prefix for all keys returned by old version of GetMapProperty. 3. Added stall values with keys starting with "io_stall." prefix Closes facebook/mysql-5.6#695 Differential Revision: D5723305 Pulled By: Tema
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This branch extends existing property map which keeps values in doubles to keep values in strings so that it can be used to provide wider range of properties. The immediate need for that is to provide IO stall stats in an easy parseable way to MyRocks which is also part of this branch.
Test Plan:
Modify the existing tests which uses doubles to use strings.
Added IO stall are tested thought MyRocks test.