CPU-MEM-monitor is a simple script to monitor Linux CPU and memory usage over time and output a CSV report that can be easily analysed and plotted with e.g., Excel or OpenOffice Calc.
The script is focused on monitoring the CPU (by thread) and memory usage (by process) of one or more Linux processes over time. This is very useful to optimize programs that need to be running for long amounts of time (e.g., servers) or to ensure that no memory leaks are present in your software. The use of regular expressions make it easy to capture CPU/memory usage of strongly-multithreaded applications. The output is written in an Excel-friendly (or OpenOffice-friendly!) .CSV file whose contents can be easily plotted with few clicks.
The script is all contained in a single file so that it is very easy to copy and move around Linux machines. It only requires BASH, TOP (or PIDSTAT) and other common utility programs (like GREP and AWK) installed.
The script is also useful to analyze a (multithreaded) process during a long amount of time (e.g., several days / months). For such usage scenario it is convenient to use the screen utility.
Command Line Interface
The script accepts the following command-line options:
Usage: ./CPU_MEM_monitor_v11.sh [-h] [-v] [--use-pidstat] [-t THREADNAME_REGEX] [-p AUX_PROCESS1] [-p AUX_PROCESS2] ... Version 1.1, by Francesco Montorsi Automates TOP/PIDSTAT monitoring and resource usage statistics logging to a .CSV file -h this help -v be verbose --use-pidstat pidstat rather than top will be used -t <tregex> monitor threads whose name match the regex <tregex>; e.g. 'mythread\|myotherthread' -p <auxproc> monitor CPU and memory usage of the auxiliary process <auxproc> Default option values: VERBOSE: false USEPIDSTAT: false THREADNAME_REGEX: myparworker\|threadtype2 AUX_PROCESS #0: multithread Output .csv file name will be automatically generated based on current date and hostname. Associated to the .csv also a .log file containing info messages will be generated. DIFFERENCES BETWEEN AUXILIARY PROCESS MONITORING AND THREAD MONITORING: Auxiliary process monitoring collects per-process CPU% and MEMORY values. Thread monitoring collects per-thread CPU% and MEMORY values. If a monitored aux process dies, the script will attempt to restart logging process completely. If a monitored thread ends, the script will do nothing and continue logging.
As an example, assume you want to monitor a process called "multithread" that generates several threads named "myparworker/0", "myparworker/1", etc or "threadtype2/0", "threadtype2/1", etc (such example program is available as an example in the "docs" folder); in this case you can launch the CPU_MEM_monitor script as:$ ./CPU_MEM_monitor.sh -t "myparworker\|operation2thread" -p multithread
which will produce:
The .csv file produced by above example can be downloaded clicking here.
The .csv file can be easily plotted to get an idea of the thread CPU usage and the per-process memory used:
How To Install
$ wget --no-check-certificate http://raw.githubusercontent.com/f18m/CPU-MEM-monitor/master/CPU_MEM_monitor.sh $ chmod a+x CPU_MEM_monitor.sh # ensure execute permissions $ sudo mv CPU_MEM_monitor.sh /usr/local/bin # make it available system-wide
How To Use
$ screen # in case you want to leave logging after closing terminal screen $ CPU_MEM_monitor.sh -t yourThreadName -p yourProcessName [CTRL+A, D] # to detach from the screen session and leave the script running
After the session of performance logging is completed, just double-click the .csv file created in the directory where the script has been launched and analyse the logged CPU/memory values.
The "decimate_compress_csv.sh" script available from:
$ wget --no-check-certificate http://raw.githubusercontent.com/f18m/CPU-MEM-monitor/master/decimate_compress_csv.sh $ chmod a+x decimate_compress_csv.sh # ensure execute permissions $ sudo mv decimate_compress_csv.sh /usr/local/bin
can be used to decimate very large .csv files (e.g., generated after several days of logging), to ease manipulation by e.g. Excel.