Performance Testing

Measuring overall duperemove performance

The simplest way to test duperemove performance is to run it, being careful to pipe all output to /dev/null.

time duperemove -rd /btrfs &> /dev/null
real    0m8.264s
user    0m8.132s
sys     0m0.135s

Measuring performance of file scan and hashing stage

We can also use hashfiles to allow for performance testing of the hashing and extent finding stages. Duperemove won't do the extent search (and therefore, dedupe) steps when writing out a hashfile. Therefore, this command will measure the time duperemove takes to scan and hash a data set. Ignore the made-up numbers that time is reporting here.

time duperemove --write-hashes=hashfile.dup -r /btrfs &> /dev/null
real    0m8.264s
user    0m8.132s
sys     0m0.135s

Measuring performance of extent search stage

Conversly, hashes can be read from a file. When doing this, duperemove skips the hashing stage (though it will try dedupe if asked). If run without dedupe, and with a hashfile argument, duperemove will only perform the extent search stage. This becomes useful for measuring performance of the extent search. Again, ignore the made-up results:

time duperemove --read-hashes=hashfile.dup &> /dev/null
real    0m8.264s
user    0m8.132s
sys     0m0.135s