Skip to content

Update Extra Verbose Stats Display #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
koolzz merged 3 commits into from
Jun 4, 2019
Merged

Update Extra Verbose Stats Display #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fc44685
Fix extra verbose stats header
koolzz Jun 4, 2019
94a083d
Fix comas, clean up defs
koolzz Jun 4, 2019
864b855
Update README.md
koolzz Jun 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions onvm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,85 @@ Packet Helper Library
--
The Packet Helper Libary provides an interface to extract TCP/IP, UDP, and other packet headers that were lost due to [Intel DPDK][dpdk]. Since DPDK avoides the Linux Kernel to bring packet data into userspace, we lose the encapsulation and decapsulation that the Kernel provides. DPDK wraps packet data inside its own structure, the `rte_mbuf`.

Stats
--

There are 2 ways to view onvm_mgr stats:

1. Console stats provide a detailed overview of all statistics and provide different verbosity modes.

The default mode displays only the most critical NF information:
```
PORTS
-----
Port 0: '90:e2:ba:b3:bc:6c'

Port 0 - rx: 4 ( 0 pps) tx: 0 ( 0 pps)

NF TAG IID / SID / CORE rx_pps / tx_pps rx_drop / tx_drop out / tonf / drop
----------------------------------------------------------------------------------------------------------------------
speed_tester 1 / 1 / 4 1693920 / 1693920 0 / 0 0 / 40346970 / 0
```

The verbose mode (`-v`) adds `PNT`(Parent ID), `S|W`(NF state, sleeping or working), `CHLD`(Children count), as well as additional packet processing information:
```
PORTS
-----
Port 0: '90:e2:ba:b3:bc:6c'

Port 0 - rx: 4 ( 0 pps) tx: 0 ( 0 pps)

NF TAG IID / SID / CORE rx_pps / tx_pps rx_drop / tx_drop out / tonf / drop
PNT / S|W / CHLD drop_pps / drop_pps rx_drop / tx_drop next / buf / ret
----------------------------------------------------------------------------------------------------------------------
speed_tester 1 / 1 / 4 9661664 / 9661664 94494528 / 94494528 0 / 94494487 / 0
0 / W / 0 0 / 0 0 / 0 0 / 0 / 128
```

The shared core mode (`-c`) adds wakeup information stats:
```
PORTS
-----
Port 0: '90:e2:ba:b3:bc:6c'

Port 0 - rx: 5 ( 0 pps) tx: 0 ( 0 pps)

NF TAG IID / SID / CORE rx_pps / tx_pps rx_drop / tx_drop out / tonf / drop
PNT / S|W / CHLD drop_pps / drop_pps rx_drop / tx_drop next / buf / ret
wakeups / wakeup_rt
----------------------------------------------------------------------------------------------------------------------
simple_forward 2 / 2 / 4 27719 / 27719 764439 / 764439 0 / 764439 / 0
0 / S / 0 0 / 0 0 / 0 0 / 0 / 0
730557 / 25344

speed_tester 3 / 1 / 5 27719 / 27719 764440 / 764439 0 / 764440 / 0
0 / W / 0 0 / 0 0 / 0 0 / 0 / 1
730560 / 25347



Shared core stats
-----------------
Total wakeups = 1461122, Wakeup rate = 50696
```

The super verbose stats mode (`-vv`) dumps all stats in a comma separated list for easy script parsing:
```
#YYYY-MM-DD HH:MM:SS,nic_rx_pkts,nic_rx_pps,nic_tx_pkts,nic_tx_pps
#YYYY-MM-DD HH:MM:SS,nf_tag,instance_id,service_id,core,parent,state,children_cnt,rx,tx,rx_pps,tx_pps,rx_drop,tx_drop,rx_drop_rate,tx_drop_rate,act_out,act_tonf,act_drop,act_next,act_buffer,act_returned,num_wakeups,wakeup_rate
2019-06-04 08:54:52,0,4,4,0,0
2019-06-04 08:54:53,0,4,0,0,0
2019-06-04 08:54:54,simple_forward,1,2,4,0,W,0,29058,29058,29058,29058,0,0,0,0,0,29058,0,0,0,0,28951,28951
2019-06-04 08:54:54,speed_tester,2,1,5,0,S,0,29058,29058,29058,29058,0,0,0,0,0,29059,0,0,0,1,28952,28952
2019-06-04 08:54:55,0,4,0,0,0
2019-06-04 08:54:55,simple_forward,1,2,4,0,W,0,101844,101843,72785,72785,0,0,0,0,0,101843,0,0,0,0,101660,101660
2019-06-04 08:54:55,speed_tester,2,1,5,0,W,0,101844,101843,72785,72785,0,0,0,0,0,101844,0,0,0,1,101660,101660
```


2. Web Stats provide an easy to navigate web view with NF performance graphs, manager port stats and the core layout across the system. It also keeps track of timestamps for NF events such as NF_STARTING and NF_STOPPING.

For more info and design details check the [web stats docs][web_stats_docs]

[dpdk]: http://dpdk.org/
[web_stats_docs]: ../onvm_web/README.md
12 changes: 5 additions & 7 deletions onvm/onvm_mgr/onvm_stats.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,11 +130,8 @@ char buffer[20];
void
onvm_stats_init(uint8_t verbosity_level) {
if (verbosity_level == ONVM_RAW_STATS_DUMP) {
printf("#YYYY-MM-DD HH:MM:SS,nic_rx_pkts,nic_rx_pps,nic_tx_pkts,nic_tx_pps\n");
printf(
"#YYYY-MM-DD "
"HH:MM:SS,instance_id,service_id,rx,tx,rx_pps,tx_pps,rx_drop,tx_drop,rx_drop_rate,tx_drop_rate,act_"
"out,act_tonf,act_drop,act_next,act_buffer,act_returned\n");
printf("%s", ONVM_STATS_RAW_DUMP_PORT_MSG);
printf("%s", ONVM_STATS_RAW_DUMP_NF_MSG);
}
}

Expand Down Expand Up @@ -341,7 +338,7 @@ onvm_stats_display_ports(unsigned difftime, uint8_t verbosity_level) {
nic_tx_pps = (nic_tx_pkts - tx_last[i]) / difftime;

if (verbosity_level == ONVM_RAW_STATS_DUMP) {
fprintf(stats_out, ONVM_STATS_ADV_PORTS, buffer,
fprintf(stats_out, ONVM_STATS_RAW_DUMP_PORTS_CONTENT, buffer,
(unsigned)ports->id[i], nic_rx_pkts, nic_rx_pps, nic_tx_pkts, nic_tx_pps);

} else {
Expand Down Expand Up @@ -495,9 +492,10 @@ onvm_stats_display_nfs(unsigned difftime, uint8_t verbosity_level) {
if (verbosity_level == ONVM_RAW_STATS_DUMP) {
fprintf(stats_out, ONVM_STATS_RAW_DUMP_CONTENT,
buffer, nfs[i].tag, nfs[i].instance_id, nfs[i].service_id, nfs[i].thread_info.core,
nfs[i].thread_info.parent, state, rte_atomic16_read(&nfs[i].thread_info.children_cnt),
rx, tx, rx_pps, tx_pps, rx_drop, tx_drop, rx_drop_rate, tx_drop_rate,
act_out, act_tonf, act_drop, act_next, act_buffer, act_returned,
num_wakeups, wakeup_rate, active);
num_wakeups, wakeup_rate);
} else if (verbosity_level == 2) {
fprintf(stats_out, ONVM_STATS_ADV_CONTENT,
nfs[i].tag, nfs[i].instance_id, nfs[i].service_id, nfs[i].thread_info.core,
Expand Down
37 changes: 22 additions & 15 deletions onvm/onvm_mgr/onvm_stats.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,33 +70,40 @@ extern const char *NF_MSG[3];
" PNT / S|W / CHLD drop_pps / drop_pps rx_drop / tx_drop next / buf / ret\n"\
" wakeups / wakeup_rt\n"\
"----------------------------------------------------------------------------------------------------------------------\n"
#define ONVM_STATS_RAW_DUMP_CONTENT \
"%s,%s,%u,%u,%u,%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64\
",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64\
",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%d\n"
#define ONVM_STATS_RAW_DUMP_PORT_MSG \
"#YYYY-MM-DD HH:MM:SS,nic_rx_pkts,nic_rx_pps,nic_tx_pkts,nic_tx_pps\n"
#define ONVM_STATS_RAW_DUMP_NF_MSG \
"#YYYY-MM-DD HH:MM:SS,nf_tag,instance_id,service_id,core,parent,state,children_cnt,"\
"rx,tx,rx_pps,tx_pps,rx_drop,tx_drop,rx_drop_rate,tx_drop_rate,"\
"act_out,act_tonf,act_drop,act_next,act_buffer,act_returned,num_wakeups,wakeup_rate\n"
#define ONVM_STATS_REG_CONTENT \
"%-14s %2u / %-2u / %2u %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64 " / %-11" PRIu64\
" %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64 " \n"
#define ONVM_STATS_REG_TOTALS \
"SID %-2u %2u%s - %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64\
" / %-11" PRIu64 " %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64 "\n"
#define ONVM_STATS_REG_PORTS \
"Port %u - rx: %9" PRIu64 " (%9" PRIu64 " pps)\t"\
"tx: %9" PRIu64 " (%9" PRIu64 " pps)\n"
#define ONVM_STATS_ADV_CONTENT \
"%-14s %2u / %-2u / %2u %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64 " / %-11" PRIu64\
" %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64\
"\n %5" PRId16 " / %c / %u %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64 " / %-11" PRIu64\
" %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64 "\n"
#define ONVM_STATS_SHARED_CORE_CONTENT \
" %11" PRIu64 " / %-11" PRIu64"\n"
#define ONVM_STATS_REG_CONTENT \
"%-14s %2u / %-2u / %2u %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64 " / %-11" PRIu64\
" %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64 " \n"
#define ONVM_STATS_ADV_TOTALS \
"SID %-2u %2u%s - %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64\
" / %-11" PRIu64 " %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64\
"\n %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64\
" / %-11" PRIu64 " %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64 "\n"
#define ONVM_STATS_REG_TOTALS \
"SID %-2u %2u%s - %9" PRIu64 " / %-9" PRIu64 " %11" PRIu64\
" / %-11" PRIu64 " %11" PRIu64 " / %-11" PRIu64 " / %-11" PRIu64 "\n"
#define ONVM_STATS_ADV_PORTS \
"%s,%u,%" PRIu64 ",%" PRIu64 ",%" PRIu64 "%" PRIu64 "\n"
#define ONVM_STATS_REG_PORTS \
"Port %u - rx: %9" PRIu64 " (%9" PRIu64 " pps)\t"\
"tx: %9" PRIu64 " (%9" PRIu64 " pps)\n"
#define ONVM_STATS_RAW_DUMP_CONTENT \
"%s,%s,%u,%u,%u,%u,%c,%u,%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64\
",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64\
",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 "\n"
#define ONVM_STATS_RAW_DUMP_PORTS_CONTENT \
"%s,%u,%" PRIu64 ",%" PRIu64 ",%" PRIu64 ",%" PRIu64 "\n"

#define ONVM_STATS_FOPEN_ARGS "w+"
#define ONVM_STATS_PATH_BASE "../onvm_web/"
#define ONVM_JSON_STATS_FILE ONVM_STATS_PATH_BASE "onvm_json_stats.json"
Expand Down