Throughput differences between dbench versions.

[sangorrin]

Hi,

Tim Bird[1] and I noticed a huge difference between the throughput results in dbench v3.04 and v4.00.
If you know the reason, could you explain it please?

Thanks,
Daniel

PS: here are a few results

1. Version 3.04 (built from Fuego)

$ sudo ./dbench -c client.txt -t 10 1
dbench version 3.04 - Copyright Andrew Tridgell 1999-2004

Running for 10 seconds with load 'client.txt' and minimum warmup 2 secs
1 clients started
1 122253 676.39 MB/sec warmup 1 sec
1 372673 677.01 MB/sec execute 1 sec
1 493684 672.06 MB/sec execute 2 sec
1 618774 672.82 MB/sec execute 3 sec
1 743963 673.17 MB/sec execute 4 sec
1 869201 673.76 MB/sec execute 5 sec
1 990318 672.74 MB/sec execute 6 sec
1 1115517 673.64 MB/sec execute 7 sec
1 1240630 673.45 MB/sec execute 8 sec
1 1366000 673.53 MB/sec execute 9 sec
1 1486908 673.17 MB/sec cleanup 10 sec
1 1486909 672.38 MB/sec cleanup 10 sec

2. Version 4.00 (Debian jessie binary)

$ sudo dbench -t 10 1
dbench version 4.00 - Copyright Andrew Tridgell 1999-2004

Running for 10 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 2 secs
0 of 1 processes prepared for launch 0 sec
1 of 1 processes prepared for launch 0 sec
releasing clients
1 801 21.48 MB/sec warmup 1 sec latency 66.724 ms
1 4362 7.05 MB/sec execute 1 sec latency 66.720 ms
1 6037 7.15 MB/sec execute 2 sec latency 58.372 ms
1 7787 8.63 MB/sec execute 3 sec latency 267.384 ms
1 8094 6.65 MB/sec execute 4 sec latency 66.724 ms
1 11327 9.00 MB/sec execute 5 sec latency 58.295 ms
1 11572 7.60 MB/sec execute 6 sec latency 66.690 ms
1 13954 8.46 MB/sec execute 7 sec latency 58.382 ms
1 14996 8.05 MB/sec execute 8 sec latency 158.601 ms
1 15833 7.68 MB/sec execute 9 sec latency 208.083 ms

3. Version built from the latest GIT commit

$ ./dbench -B fileio -c loadfiles/client.txt -t 10 1
dbench version 4.00 - Copyright Andrew Tridgell 1999-2004

Running for 10 seconds with load 'loadfiles/client.txt' and minimum warmup 2 secs
Waiting for child processes to finish setup.
Releasing clients
1 873 21.66 MB/sec warmup 1 sec latency 58.935 ms
1 4362 5.23 MB/sec execute 1 sec latency 58.419 ms
1 6037 6.24 MB/sec execute 2 sec latency 58.491 ms
1 7841 8.06 MB/sec execute 3 sec latency 66.783 ms
1 8995 7.32 MB/sec execute 4 sec latency 58.433 ms
1 11332 8.63 MB/sec execute 5 sec latency 82.462 ms
1 11575 7.30 MB/sec execute 6 sec latency 67.725 ms
1 14872 8.90 MB/sec execute 7 sec latency 58.217 ms
1 15079 7.85 MB/sec execute 8 sec latency 66.710 ms
1 17389 8.38 MB/sec execute 9 sec latency 66.710 ms
1 cleanup 10 sec
0 cleanup 10 sec

[1] Here is a table of results provided by Tim Bird as well (https://lists.linuxfoundation.org/pipermail/fuego/2018-April/001838.html):
test spec board tguid result

dbench3 default bbb <<< wouldn't compile >>>
dbench3 default bbb dbench3.Throughput 11.5173
dbench4 default bbb dbench.Throughput 0.0282559
dbench4 roota bbb dbench.Throughput 0.116864

dbench testdir docker dbench.Throughput 1741.26
dbench testdir docker dbench.Throughput 1831.78
dbench3 default docker dbench3.Throughput 1169.77
dbench4 default docker dbench.Throughput 10.394
dbench4 roota docker dbench.Throughput 10.959

dbench default min1 dbench.Throughput 254.03
dbench3 default min1 dbench3.Throughput 253.83
dbench3 default min1 dbench3.Throughput 251.732
dbench4 default min1 dbench.Throughput 32.1764
dbench4 default min1 dbench.Throughput 33.0374
dbench4 roota min1 dbench.Throughput 34.0026
dbench4 roota min1 dbench.Throughput 21.3904

dbench default ren1 dbench.Throughput 8.41553
dbench default ren1 dbench.Throughput 8.62088
dbench3 default ren1 dbench3.Throughput 8.43887
dbench3 default ren1 dbench3.Throughput 8.45205
dbench4 default ren1 dbench.Throughput 7.53374
dbench4 roota ren1 dbench.Throughput 8.03552
dbench4 roota ren1 dbench.Throughput 7.72424

dbench default rpi3-1 dbench.Throughput 136.099
dbench default rpi3-1 dbench.Throughput 132.869
dbench3 default rpi3-1 dbench3.Throughput 159.953
dbench4 default rpi3-1 dbench.Throughput 39.9088
dbench4 roota rpi3-1 dbench.Throughput 50.6633