Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Simple benchmarking framework #6733
Benchmarking framework, loosely based on google's micro-benchmarking
Wny not use the Google Benchmark framework? Because adding Even More Dependencies
The benchmark framework is hard-coded to run each benchmark for one wall-clock second,
See src/bench/Examples.cpp for a sanity-test benchmark that just benchmarks
To compile and run benchmarks:
Removed the $(CURL_LIBS) noticed by @jonasschnelli.
And added the new libevent and libzmq dependencies.
I also decided to make it disabled by default, so you have to configure --enable-bench, so it doesn't slow down compile times for everybody.
@laanwj : RE: gettimeofday overhead and timing things that are very fast: timing how fast gettimeofday() is and then warning the developer if their benchmarking code runs into that overhead (maybe suggest that they loop 1,000 times inside the KeepRunning loop) would be another way of preventing that problem. Feel free to improve, I'm planning on using this to benchmark things that take microseconds to run, not nanoseconds.
@jgarzik : I KNEW you were going to say that...
Would a Travis configuration that runs benchmarks be good enough to prevent code rot? Compiling and linking Yet Another Binary that approximately nobody will run (unless you're actively working on optimizing something) tickles my "features shouldn't cost anything unless you're using them" sensibility.
I think it should stay disabled by default as it is now, because benchmarking will be done on purpose by developers only.
Concept ACK. I like it.
This is a way to benchmark parts of code. Now we should identify parts that need some love and prepare benchmarks for them specifically. With this framework, people can report speed fixes with proper benchmark code which can be tested before and after applying the fix.
Fixed @paveljanik 's spaces nits, and removed the unneeded SelectParams inherited from the unit test code.
Data on the "compile by default or not" decision:
It takes 8 seconds on my machine to compile and link the benchmarking code, single-processor, ccache cleared.
Damn you @laanwj, you inspired me to spend another hour implementing support for really fast benchmarks....
Picked @paveljanik 's CreateNewBlock nit (good catch, I copied this from a CreateNewBlock benchmark I haven't finished). I decided to rename MilliSleep.cpp to Examples.cpp, and added a "see how fast sin() runs" benchmark to test the really-fast-benchmark support (works nicely).
And switched back to compile-by-default, which seems to be the consensus.
I'm really and truly done tweaking this now, assuming Travis is happy.