A Benchmark program to test Hard Drives, SSD Drives, HBAs, RAID Adapters & Storage Controllers
Use this Benchmark program to test any disk physical or virtual, local or remote. This is a very light C program with the following features:
- Multi-threaded: can simulate concurrent read/writes from many threads
- Very fast: it utilizes lower system I/O
read()/write()c-lib functions operating directly with buffers on a file descriptor. No JVM not even formatted - read/write. Nothing is faster than this. Can simulate easily MySQL I/O Operations, FTP/SFTP servers, Streaming Servers, Busy HTTP servers
- Supports many features, like random file size generation, random sleep time before read writes
- Supports write only mode
- Supports buffer size configuration during the test
- You can also preserve the files to check what has been written from threads.
- Per thread debug level on what has been written/read from disk
- In the end prints the summaries and average times from all threads.
- Prints some system level hints like buffer sizes
- Output CPU and Wall time for the operation timing.
How it works
- You specify number of threads, thread read/write repeat times, file sizes
- Program runs the thread, each of them performing times configured
- Each thread average and total times are printed
- Program stops threads and gather summary results
How to run
Run it without arguments to get the option lists:
./disk-benchmark -p -t -r -b -l -u -a Makes a disk benchmark on by creating threads (default is 5 threads)) Each thread first writes and then reads a file of size between and OR for times. Thread is reading/ writing the file using I/O buffer of bytes The resulting Time for Reading/Writing the file for each thread and summary results are printed. Other options -e Make each thread sleep for before attempt to read. -E : Make each thread sleep for randomly picked sec between : before attempt to read. -w Make each thread do write only. -n Make each thread NOT delete the sample files. MAKE SURE YOU HAVE ENOUGH SPACE if you turn this option! -B : Make each thread write/read files with those buffer sizes. -x print values only. Use this to grab values for spreadsheets. -o Don't print clocks. -j Don't print scenario information. -h Don't print headers. -d print system local time also. -c work continuously. Blocks in a loop recreating threads without printing summaries. You have to interrupt with Cntrl-C -z Sleep interval time in seconds when work continuously. Setting this to something small, will default to high CPU load. ./disk-benchmark -s Prints information around system I/O. Use this info to set R/W buffer size for your tests with -b option. Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite. Returns 0 on OK , 1 on Error. Author: Michael Mountrakis 2016 - email@example.com Version v1.1 built on 15/4/2016
- Thread number is 5
- Buffer size for Read/Write set to system´s
- File size minimum is 1024 Bytes
- File size maximum is 10 KB
Running a test
# ./disk-benchmark -p /monsoon/contentstore/ -t 10 -l 10000 -u 90000 -r 5 -E 2:10 path=/monsoon/contentstore/ threads=10 sleep sec between write/read = 0 repeats per thread=5 random pick sleep sec from [2 10] Lower file size=10000 Upper file size=90000 Absolute file size=0 Read/Write buffer size=8192 Do write only=0 Delete files=1 Avg W=0.000295 Avg R=0.000121 Total W=0.001475 Total R=0.000603 Total Time=0.002078 Sleep=4.400000 Avg File Size =38878.000000 Avg W=0.000173 Avg R=0.001933 Total W=0.000863 Total R=0.009665 Total Time=0.010528 Sleep=4.800000 Avg File Size =59948.400000 Avg W=0.000124 Avg R=0.001928 Total W=0.000618 Total R=0.009640 Total Time=0.010258 Sleep=5.200000 Avg File Size =39912.000000 Avg W=0.000388 Avg R=0.000107 Total W=0.001942 Total R=0.000536 Total Time=0.002479 Sleep=6.000000 Avg File Size =61133.600000 Avg W=0.000143 Avg R=0.000090 Total W=0.000713 Total R=0.000452 Total Time=0.001166 Sleep=6.000000 Avg File Size =53212.000000 Avg W=0.000154 Avg R=0.000090 Total W=0.000771 Total R=0.000449 Total Time=0.001220 Sleep=6.200000 Avg File Size =50201.000000 Avg W=0.000344 Avg R=0.000117 Total W=0.001719 Total R=0.000584 Total Time=0.002303 Sleep=6.400000 Avg File Size =53158.000000 Avg W=0.000249 Avg R=0.002070 Total W=0.001246 Total R=0.010352 Total Time=0.011599 Sleep=6.400000 Avg File Size =74144.400000 Avg W=0.000303 Avg R=0.000112 Total W=0.001515 Total R=0.000561 Total Time=0.002075 Sleep=7.600000 Avg File Size =55071.200000 Avg W=0.000114 Avg R=0.000093 Total W=0.000568 Total R=0.000465 Total Time=0.001033 Sleep=7.800000 Avg File Size =55898.600000 Summaries - Averages all threads Avg W=0.000229 Avg R=0.000666 Total W=0.001143 Total R=0.003331 Total Time=0.004474 Sleep=6.080000 Avg File Size =54155.720000 Wall time 39.000000, CPU time 0.010000 Finished
In the first section it prints out your option. From what you can see in the example:
- 10 Threads are utilized
- Every thread will write and then read a file of random size between 10000 to 90000 bytes for 5 times
- Every thread will sleep for a random time between 2 to 10 seconds before it reads the file
In the second section, when all threads finish it will print the following for each thread:
Avg Wtime in seconds: The time needed to
write()the file 5 times
Avg Rtime in seconds: The time needed to
read()the file 5 times
Total Wtime in seconds: The time needed to write the file 5 times
Total Rtime in seconds: The time needed to read the file 5 times
Total Timetime in seconds: The time needed for write and read the file 5 times
Sleeptime in seconds: The average time the thread went to sleep during writing / reading the file for 5 times
Avg File Size: average size in bytes of the file each of the thread created for 5 times
In the last section, when all threads finish the all threads performance summary is printed:
Avg Wtime in seconds: The average time needed for a single
write()operation of all 10 threads
Avg Rtime in seconds: The average time needed for a single
read()operation of all 10 threads
Total Wtime in seconds: The average total time needed to write the file 5 times from all 10 threads
Total Rtime in seconds: The average total time needed to read the file 5 times from all 10 threads
Total Timetime in seconds: The average total time needed to write and read the file 5 times from all 10 threads
Sleeptime in seconds: The average time all threads went to sleep during writing / reading the file for 5 times
Avg File Size: average size in bytes of all files created by each thread
Also timing of the overall operation
Wall timeis the number of seconds the entire operation lasted
CPU timeis the number of seconds the program run inside CPU - without calculating the thread sleep time.
Examples of usage
Print I/O info
# ./disk-benchmark -s System I/O Block Site (stat.h : struct stat st_blksize) is 4096 bytes System Buffer Size (stdio.h : BUFSIZ) is 8192 bytes
Run a test of 10 threads each writing a file of size 100 * BUFSIZ with the default buffer size without reading the file. Each thread should sleep for some seconds randomly picked from the interval [2,10] sec.
# ./disk-benchmark -p /monsoon/contentstore/ -t 10 -a 81920 -E 2:10 -w
Run a test of 30 threads each writing a file of size between the interval [10000,90000] bytes with a buffer size of 100 bytes with subsequent reading the file. Each thread should sleep for some seconds randomly picked from the interval [2,10] sec.
# ./disk-benchmark -p /monsoon/contentstore/ -t 40 -l 10000 -u 90000 -b 100 -E 2:10
Run a continuous test of 10 Threads each writing and reading a file of 2MB on path /contentstore. Each time a thread will pause for 1 sec before read the file. Program will re-run every 6 seconds. Dont print headers, test info and output values only.
./disk-benchmark -p /contentstore -t 10 -a 2000000 -r 20 -x -h -j -z 6 -c
Doing stupid things like:
# ./disk-benchmark -p /monsoon/contentstore/ -t 100 -a 90000000 -n -c -z 0 -b 100
In other words, put the benchmark to work continuously with 100 threads writing 90MB files without deleting them and no sleep interval using a buffer or 100 bytes.
Commands like this will kill your system in seconds especially when run as root. This particular example will:
- Full the disk under test in seconds
- Cause huge CPU load uIn simple words: Use this test tool at your own sake and risk!!!
Test native Monsoon 2.0 ESX volumes:
Lets pick the / partition of a Monsoon 2.0 image:
# df -h Filesystem Size Used Avail Use% Mounted o /dev/mapper/vg0-root 32G 1.1G 29G 4% /
Scenario: We will create 10 threads each of them will write and then read a file of size 10.000.000 Bytes ( almost 10 MB) for 20 times.
# ./disk-benchmark -p /root/tmp -t 10 -a 10000000 -r 20 path=/root/tmp threads=10 sleep sec between write/read = 0 repeats per thread=20 random pick sleep sec from [0 0 Lower file size=1024 Upper file size=10240 Absolute file size=10000000 Read/Write buffer size=8192 Do write only=0 Delete files=1 Avg W=0.045533 Avg R=0.019135 Total W=0.910665 Total R=0.382699 Total Time=1.293364 Sleep=0.000000 Avg File Size =10000000.00000 Avg W=0.044116 Avg R=0.025437 Total W=0.882325 Total R=0.508733 Total Time=1.391058 Sleep=0.000000 Avg File Size =10000000.00000 Avg W=0.049421 Avg R=0.018219 Total W=0.988420 Total R=0.364379 Total Time=1.352800 Sleep=0.000000 Avg File Size =10000000.00000 .. Avg W=0.055249 Avg R=0.021643 Total W=1.104971 Total R=0.432856 Total Time=1.537826 Sleep=0.000000 Avg File Size =10000000.00000 Avg W=0.054114 Avg R=0.020706 Total W=1.082273 Total R=0.414117 Total Time=1.496390 Sleep=0.000000 Avg File Size =10000000.000000 Summaries - Averages all thread Avg W=0.050939 Avg R=0.018539 Total W=1.018786 Total R=0.370771 Total Time=1.389556 Sleep=0.000000 Avg File Size =10000000.00000 Wall time 2.000000, CPU time 3.51000 Finished
Test ESX extended volumes
This is a test on a logical volume created from 2 ESX mounted native volumes.
# This is the logical volume # df -h Filesystem Size Used Avail Use% Mounted o /dev/mapper/vol_grp1-logical_vol1 40G 2.4G 35G 7% /monsoon/contentstore # this volume is composed from volume group vol_grp # lvdisplay /dev/vol_grp1/logical_vol --- Logical volume -- LV Path /dev/vol_grp1/logical_vol LV Name logical_vol VG Name vol_grp LV UUID V66W74-SPJc-F2Sw-Tfqc-hV0Z-HS0I-bMqZN LV Write Access read/writ LV Creation host, time mo-0bcd54f94, 2016-04-11 17:43:22 +000 LV Status availabl # open LV Size 39.99 GiB Current LE 10238 Segments Allocation inheri Read ahead sectors aut - currently set to 819 Block device 253: # This group vol_grp1 consists of the following physical volumes - each of 20 GB # pvsca PV /dev/sdb VG vol_grp1 lvm2 [20.00 GiB / 0 free PV /dev/sdc VG vol_grp1 lvm2 [20.00 GiB / 0 free Total: 3 [99.70 GiB] / in use: 3 [99.70 GiB] / in no VG: 0 [0 ]
Test the volume with the previous scenario
# ./disk-benchmark -p /monsoon/contentstore -t 10 -a 10000000 -r 20 ... Summaries - Averages all threadssAvg W=0.023361 Avg R=0.095913 Total W=0.467228 Total R=1.918250 Total Time=2.385478 Sleep=0.000000 Avg File Size =10000000.000000 Wall time 4.000000, CPU time 5.100000
This test will be done to a replicated GLusterFS file system that is mounted from a remote device.
ha-ma-rot1 root@mo-0bcd54f94:~ # gluster volume inf Volume Name: contentstor Type: Replicat Volume ID: d300220c-d45b-41bc-94a6-47becc0cfa4 Status: Starte Number of Bricks: 1 x 2 = Transport-type: tc Bricks Brick1: mo-0bcd54f94.mo.sap.corp:/monsoon/contentstor Brick2: mo-4543f2f78.mo.sap.corp:/monsoon/contentstor ha-ma-rot1 root@mo-0bcd54f94:~ # mkdir /contentstor ha-ma-rot1 root@mo-0bcd54f94:~ # mount -t glusterfs mo-4543f2f78.mo.sap.corp:/contentstore /contentstore ha-ma-rot1 root@mo-0bcd54f94:~ # mount | grep cont /dev/mapper/vol_grp1-logical_vol1 on /monsoon/contentstore type ext4 (rw,relatime,data=ordered mo-4543f2f78.mo.sap.corp:/contentstore on /contentstore type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) ha-ma-rot1 root@mo-0bcd54f94:~ # df -h | grep con /dev/mapper/vol_grp1-logical_vol1 40G 49M 38G 1% /monsoon/contentstor mo-4543f2f78.mo.sap.corp:/contentstore 40G 49M 38G 1% /contentstor ha-ma-rot1 root@mo-0bcd54f94:~ # ./disk-benchmark -p /contentstore -t 10 -a 10000000 -r 2 Summaries - Averages all threads Avg W=1.742522 Avg R=0.293959 Total W=34.850432 Total R=5.879178 Total Time=40.729610 Sleep=0.000000 Avg File Size =10000000.00000 Wall time 42.000000, CPU time 5.5100
How to builtd
gcc, the GNU C Compiler
# git clone https://github.com/illumine/disk-benchmark # cd disk-benchmark/src/ # gcc disk-benchmark.c -o disk-benchmark -l pthread -lrt -O3 -Wall # ls -l disk-benchmark -rwxr-xr-x 1 root root 23365 Apr 15 10:23 disk-benchmark
Depends only on POSIX Threads library
Built and tested with
gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC)
Windows: It compiles and run under Ming/gcc but dos not print results. Does not work.
Michael Mountrakis mountrakis.AT.illumine.gr