tools/biolatpcts: Add the example file and man page

man/man8/biolatpcts.8

@@ -0,0 +1,89 @@
+.TH biolatpcts 8  "2020-04-17" "USER COMMANDS"
+.SH NAME
+biolatpcts \- Monitor IO latency distribution of a block device.
+.SH SYNOPSIS
+.B biolatpcts [\-h] [\-i INTERVAL] [\-w which] [\-p PCT,...] [\-j] [\-v] DEV
+.SH DESCRIPTION
+
+biolatpcts traces block device I/O (disk I/O) of the specified device, and
+calculates and prints the completion latency distribution percentiles per IO
+type periodically. Example:
+
+ # biolatpcts /dev/nvme0n1
+ nvme0n1    p1    p5   p10   p16   p25   p50   p75   p84   p90   p95   p99  p100
+ read     95us 175us 305us 515us 895us 985us 995us 1.5ms 2.5ms 3.5ms 4.5ms  10ms
+ write     5us   5us   5us  15us  25us 135us 765us 855us 885us 895us 965us 1.5ms
+ discard   5us   5us   5us   5us 135us 145us 165us 205us 385us 875us 1.5ms 2.5ms
+ flush     5us   5us   5us   5us   5us   5us   5us   5us   5us 1.5ms 4.5ms 5.5ms
+ [...]
+
+biolatpcts prints a number of pre-set latency percentiles in tabular form
+every three seconds. The interval can be changed with the \-i option.
+
+The latency is measured between issue to the device and completion. The
+starting point can be changed with the \-w option.
+
+Any number of percentiles can be specified using the \-p option. The input
+percentile string is used verbatim in the output to ease machine consumption.
+
+\-j option enables json output. The result for each interval is printed on a
+single line.
+
+This tool works by tracing blk_account_io_done() with kprobe and bucketing the
+completion latencies into percpu arrays. It may need updating to match the
+changes to the function.
+
+Since this uses BPF, only the root user can use this tool.
+.SH REQUIREMENTS
+CONFIG_BPF, CONFIG_KPROBES and bcc.
+.SH OPTIONS
+\-h
+Print usage message.
+.TP
+\-i INTERVAL, \-\-interval INTERVAL
+Report interval. (default: 3)
+.TP
+\-w {from\-rq\-alloc,after\-rq\-alloc,on\-device}, \-\-which {from\-rq\-alloc,after\-rq\-alloc,on\-device}
+Which latency to measure. (default: on-device)
+.TP
+\-p PCT,..., \-\-pcts PCT,...
+Percentiles to calculate. (default: 1,5,10,16,25,50,75,84,90,95,99,100)
+.TP
+\-j, \-\-json
+Output in json.
+.TP
+\-v, \-\-verbose
+Enable debug output.
+.TP
+DEV
+Target block device. /dev/DEVNAME, DEVNAME or MAJ:MIN.
+.SH EXAMPLES
+.TP
+Print sda's I/O latency percentiles every second
+#
+.B biolatpcts \-i 1 sda
+.TP
+Print nvme0n1's all-9 I/O latency percentiles in json every second
+#
+.B biolatpcts \-p 99,99.9,99.99,99.999 \-j \-i 1 /dev/nvme0n1
+.SH OVERHEAD
+This traces kernel functions and maintains in-kernel per-cpu latency buckets,
+which are asynchronously copied to user-space. This method is very efficient,
+and the overhead for most storage I/O rates should be negligible. If you have
+an extremely high IOPS storage device, test and quantify the overhead before
+use.
+.SH SOURCE
+This is from bcc.
+.IP
+https://github.com/iovisor/bcc
+.PP
+Also look in the bcc distribution for a companion _examples.txt file containing
+example usage, output, and commentary for this tool.
+.SH OS
+Linux
+.SH STABILITY
+Unstable - in development.
+.SH AUTHOR
+Tejun Heo
+.SH SEE ALSO
+biolatency(8), biosnoop(8)

tools/biolatpcts_example.txt

@@ -0,0 +1,68 @@
+Demonstrations of biolatpcts, the Linux eBPF/bcc version.
+
+
+biolatpcts traces block device I/O (disk I/O), and prints the latency
+percentiles per I/O type. Example:
+
+# ./biolatpcts.py /dev/nvme0n1
+nvme0n1    p1    p5   p10   p16   p25   p50   p75   p84   p90   p95   p99  p100
+read     95us 175us 305us 515us 895us 985us 995us 1.5ms 2.5ms 3.5ms 4.5ms  10ms
+write     5us   5us   5us  15us  25us 135us 765us 855us 885us 895us 965us 1.5ms
+discard   5us   5us   5us   5us 135us 145us 165us 205us 385us 875us 1.5ms 2.5ms
+flush     5us   5us   5us   5us   5us   5us   5us   5us   5us 1.5ms 4.5ms 5.5ms
+[...]
+
+Unless changed with the -i option, the latency percentiles are printed every 3
+seconds.
+
+
+Any number of custom percentiles can be requested with the -p option:
+
+# ./biolatpcts.py /dev/nvme0n1 -p 01,90.0,99.9,99.99,100.0
+
+nvme0n1    01  90.0  99.9 99.99 100.0
+read      5us 4.5ms  16ms  22ms  26ms
+write    15us 255us 365us 515us 2.5ms
+discard     -     -     -     -     -
+flush     5us   5us   5us   5us  24ms
+[...]
+
+Note that the target percentile strings are preserved as-is to facilitate
+exact key matching when the output is consumed by another program.
+
+
+When the output is consumed by another program, parsing can be tedious. The -j
+option makes biolatpcts output results in json, one line per interval.
+
+# ./tools/biolatpcts.py /dev/nvme0n1 -j
+{"read": {"1": 2.5e-05, "5": 3.5e-05, "10": 4.5e-05, "16": 0.000145, "25": 0.000195, "50": 0.000355, "75": 0.000605, "84": 0.000775, "90": 0.000965, "95": 0.0015, "99": 0.0025, "100": 0.0235}, "write": {"1": 5e-06, "5": 5e-06, "10": 5e-06, "16": 5e-06, "25": 1.5e-05, "50": 2.5e-05, "75": 4.5e-05, "84": 7.5e-05, "90": 0.000195, "95": 0.000665, "99": 0.0015, "100": 0.0035}, "discard": {"1": 0.0, "5": 0.0, "10": 0.0, "16": 0.0, "25": 0.0, "50": 0.0, "75": 0.0, "84": 0.0, "90": 0.0, "95": 0.0, "99": 0.0, "100": 0.0}, "flush": {"1": 0.0, "5": 0.0, "10": 0.0, "16": 0.0, "25": 0.0, "50": 0.0, "75": 0.0, "84": 0.0, "90": 0.0, "95": 0.0, "99": 0.0, "100": 0.0}}
+[...]
+
+
+By default biolatpcts measures the duration each IO was on the device. It can
+be changed using the -w option.
+
+
+USAGE message:
+
+usage: biolatpcts.py [-h] [-i INTERVAL]
+                     [-w {from-rq-alloc,after-rq-alloc,on-device}]
+                     [-p PCT,...] [-j] [--verbose]
+                     DEV
+
+Monitor IO latency distribution of a block device
+
+positional arguments:
+  DEV                   Target block device (/dev/DEVNAME, DEVNAME or MAJ:MIN)
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -i INTERVAL, --interval INTERVAL
+                        Report interval (default: 3)
+  -w {from-rq-alloc,after-rq-alloc,on-device}, --which {from-rq-alloc,after-rq-alloc,on-device}
+                        Which latency to measure (default: on-device)
+  -p PCT,..., --pcts PCT,...
+                        Percentiles to calculate (default:
+                        1,5,10,16,25,50,75,84,90,95,99,100)
+  -j, --json            Output in json (default: False)
+  --verbose, -v