Cloudstat reads tcp per connection statistics from kernel via /proc file. TCP per connection statistics is generated by open source kernel module tcp_probe . tcp_probe module uses kprobe framework to set trace point on "tcp_rcv_established" kernel routine that reads each tcp receive packets.
Kernel driver makes TCP per connection statistics available via /proc file. TCP statistics include: connection latency (RTT), throughput (bytes transferred), packet retransmission and additional stats. Each line read via /proc file has a unique quadruplet: Source IP address, Source Port, Destination IP address and Destination Port and relevant tcp statistics.
Python agent reads tcp per connection stats via /proc/net/tcpprobe and appends it into Queue data structure with a timestamp. Another thread that blocks on HTTP request reads these connection stats from the Queue and delivers it as a JSON object.
java script (agularjs) converts JSON objects into D3 graphs that can be used to view it in browser.
cloudstat live demo - Click Start/Stop button
Compile tcpprobe driver. Makefile is provided in tcpprobe directory
- cd tcpprobe
- sudo /sbin/insmod tcp_probe_plus.ko full=1 probetime=100 bufsize=8192
- chmod 644 /proc/net/tcpprobe
Install all required python packages on cloud instance or host that you like to monitor:
- sudo apt-get update
- sudo apt-get -y install python-pi
- sudo pip install -U pip
- sudo -H pip install Django==1.6.2
- sudo pip install -U datautil
Start the server
- $python manage.py runserver hostname:port
- example: python manage.py runserver ec2-52-8-155-144.us-west-1.compute.amazonaws.com:7415
Enter hostname or IP address with port number
- example: ec2-52-8-155-144.us-west-1.compute.amazonaws.com:7415