show engine status

anirbanr-fb edited this page Aug 26, 2016 · 6 revisions
Clone this wiki locally

Syntax

SHOW ENGINE ROCKSDB STATUS;

Overview

SHOW ENGINE ROCKSDB STATUS provides useful diagnostics and debugging information regarding the RocksDB engine underneath MyRocks.

Sample Output

*************************** 1. row ***************************
  Type: DBSTATS
  Name: rocksdb
Status: 
** DB Stats **
Uptime(secs): 488870.7 total, 20.9 interval
Cumulative writes: 390M writes, 2063M keys, 390M batches, 1.0 writes per batch, ingest: 221.35 GB, 0.46 MB/s
Cumulative WAL: 390M writes, 1114 syncs, 349971.48 writes per sync, written: 221.35 GB, 0.46 MB/s
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 18K writes, 84K keys, 18K batches, 1.0 writes per batch, ingest: 10.13 MB, 0.48 MB/s
Interval WAL: 18K writes, 0 syncs, 18350.00 writes per sync, written: 0.01 MB, 0.48 MB/s
Interval stall: 00:00:0.000 H:M:S, 0.0 percent
** Level 0 read latency histogram (micros):
Count: 1787492 Average: 11.4291  StdDev: 108.22
Min: 0  Median: 9.4120  Max: 61749
Percentiles: P50: 9.41 P75: 11.31 P99: 32.69 P99.9: 172.39 P99.99: 2242.80
------------------------------------------------------
[       0,       1 )      647   0.036%   0.036% 
[       1,       2 )      730   0.041%   0.077% 
[       2,       3 )     3648   0.204%   0.281% 
[       3,       4 )    73864   4.132%   4.413% #
[       4,       5 )   193527  10.827%  15.240% ##
[       5,       6 )   145274   8.127%  23.367% ##
[       6,       7 )    86239   4.825%  28.192% #
[       7,       8 )   116563   6.521%  34.713% #
[       8,       9 )   173017   9.679%  44.392% ##
[       9,      10 )   243268  13.609%  58.002% ###
[      10,      12 )   463351  25.922%  83.924% #####

...

** Level 3 read latency histogram (micros):
Count: 3509426 Average: 11.7251  StdDev: 149.77
Min: 0  Median: 9.3439  Max: 119592
Percentiles: P50: 9.34 P75: 11.16 P99: 39.76 P99.9: 245.03 P99.99: 2689.37
------------------------------------------------------
[       0,       1 )     1307   0.037%   0.037% 
[       1,       2 )     1862   0.053%   0.090% 
[       2,       3 )     4988   0.142%   0.232% 
[       3,       4 )   184977   5.271%   5.503% #
[       4,       5 )   434340  12.376%  17.880% ##
[       5,       6 )   319563   9.106%  26.986% ##
[       6,       7 )   161419   4.600%  31.585% #
[       7,       8 )   179738   5.122%  36.707% #
[       8,       9 )   299109   8.523%  45.230% ##
[       9,      10 )   486729  13.869%  59.099% ###

...

*************************** 2. row ***************************
  Type: CF_COMPACTION
  Name: __system__
Status: 
** Compaction Stats [__system__] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      2/0       1.45   0.5      0.0     0.0      0.0       0.2      0.2       0.0   0.0      0.0      8.9        21       324    0.066       0      0
  L6      1/0       3.32   0.0      0.5     0.5      0.0       0.3      0.3       0.0   0.6     55.2     32.6         8        81    0.104   8473K  1713K
 Sum      3/0       4.77   0.0      0.5     0.5      0.0       0.5      0.5       0.0   2.4     15.6     15.6        30       405    0.074   8473K  1713K
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Uptime(secs): 488870.7 total, 488870.7 interval
Flush(GB): cumulative 0.186, interval 0.000
Cumulative compaction: 0.46 GB write, 0.00 MB/s write, 0.46 GB read, 0.00 MB/s read, 30.0 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count

*************************** 3. row ***************************
  Type: CF_COMPACTION
  Name: cf_assoc
Status: 
** Compaction Stats [cf_foo] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      3/0     192.06   0.8      0.0     0.0      0.0      13.8     13.8       0.0   0.0      0.0     44.6       318       218    1.458       0      0
  L3      6/0     110.89   0.8     19.7    19.7      0.0      18.3     18.3       0.0   0.9     50.8     47.1       398        54    7.366    313M    16M
  L4     68/0    1357.86   1.0     52.7    12.2     40.5      51.0     10.5       0.0   4.2     29.2     28.2      1851       387    4.782    907M    19M
  L5    680/0   13671.01   1.0     98.5    10.6     87.9      90.9      3.0       0.0   8.6     24.8     22.9      4064       328   12.390   3794M    55M
  L6   4650/0   137237.52   0.0     28.2     2.9     25.3      26.1      0.8       0.0   9.0      5.0      4.7      5723        89   64.309   2303M    64M
 Sum   5407/0   152569.33   0.0    199.2    45.4    153.7     200.1     46.4       0.0  14.5     16.5     16.6     12354      1076   11.481   7319M   154M
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
Uptime(secs): 488870.7 total, 488870.7 interval
Flush(GB): cumulative 13.829, interval 0.000
Cumulative compaction: 200.13 GB write, 0.42 MB/s write, 199.17 GB read, 0.42 MB/s read, 12353.9 seconds
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
...

*************************** 10. row ***************************
  Type: Memory_Stats
  Name: rocksdb
Status: 
MemTable Total: 4526099752
MemTable Unflushed: 433590984
Table Readers Total: 0
Cache Total: 12874805627
Default Cache Capacity: 0

There are 4 distinct parts of the RocksDB engine status.

DB Stats

This section provides information on overall statistics of the databases. Most of the fields are self explanatory but are listed here.

  • Uptime - in seconds, the total time the system has been up.
  • Interval refers to the length of the last statistics collection period.

  • Write Status - Total number of write batches, the total number of keys in those batches, the size of the writes and the write throughput are reported.

  • WAL - On a PUT, the record will also be written to Write Ahead Log. Total number of writes to WAL, the throughput and the number of fsync/fdatasync that happened are also reported.
  • Keys - Total number of keys in the RocksDB engine.
  • Ingest - Total Data ingested and ingestion rate in MBytes per second.
  • Average number of Writes per sync for WAL ( this gives a rough idea of how many writes might be lost on a crash ).

There is information also on stalls encountered in the engine. The above statistics are reported on a cumulative and interval basis.

Read Latency section

This section lists the latency of reads encountered for the different levels of the LSM in RocksDB (Log Structured Merge Tree).

Typical latencies can be in the low microseconds, as keys are found in the memtable and block cache, but if there is a cache miss or a key is found in lower levels, the latencies go up. This is because more number of SST files will need to be read, and more binary searches need to be done.

Column Family Compaction Statistics section.

There is one set of statistics for each column family, including the system column family. Please refer to RocksDB wiki for more information on LSM and Column Families.

The first subsection lists the count and size of the files @ each level for the Column family.

  • Rn(GB) - refers to total read at non output level
  • Rnp1(GB) - refers to read volume at output level.

Followed the levelized output, is general compaction statistics for that column family.

Memory statistics section

  • MemTable Unflushed - Size of memtable that has not been flushed to SST files.
  • Cache Total - Size of block cache