dr2chase edited this page Sep 7, 2018 · 8 revisions

These are benchmarks collected from the community used to measure the effects of changes to the Go core (compiler, runtime, garbage collector, and libraries). They should have the following properties:

  • they matter; someone cares, perhaps in a dollars-and-cents way, that they run well
  • they are go-gettable and don't require customized steps for building the benchmark
  • they run under go test -bench ...
  • they run relatively quickly, ideally a single "run" takes less than a second (there should perhaps be a separate set of longer-running benchmarks)
  • their timings are not gratuitously noisy
  • they run cleanly in a restricted environment, such as a Docker or rkt container
  • they're not gratuitously redundant with other benchmarks already in the list; we don't need ten microbenchmarks of Go transcendental functions

These benchmarks change over time, and that is okay. Their intended use is for performance testing of proposed changes; is the geometric mean better, were any benchmarks made substantially worse?

Information for each benchmark includes (or should include):

  • a short name for the benchmark
  • the path to go get the benchmark
  • a regexp for the benchmark suite excluding individual benchmarks that might be noisy, long-running, or redundant
  • (ideally) a contact person for questions about the benchmarks
short name notes go get path benchmark regexp contact
ajstarks_deck_generate Benchmark(Polygon|Arc)
benhoyt_goawk BenchmarkR
bindata Benchmark
capnproto2 Benchmark(TextMovementBetweenSegments|Growth_MultiSegment)
cespare_mph BenchmarkBuild
cespare_xxhash BenchmarkHashes/xxhash-string,n=10_MB
ericlagergren_decimal BenchmarkPi_decimal_Go_9
ethereum_bitutil Benchmark(BaseTest2KB|FastTest2KB|Encoding4KBVerySparse)
ethereum_core BenchmarkChainRead_full_10k
ethereum_corevm BenchmarkOpDiv128
ethereum_ecies BenchmarkGenSharedKeyP256
ethereum_ethash BenchmarkHashimotoLight
ethereum_sha3 BenchmarkSha3_224_MTU
ethereum_storage BenchmarkJoin_8
ethereum_trie Benchmark
gonum_blas_native Benchmark(DasumMediumUnitaryInc|Dnrm2MediumPosInc)
gonum_community BenchmarkLouvainDirectedMultiplex
gonum_lapack_native BenchmarkDgeev/Circulant10
gonum_mat Benchmark(MulWorkspaceDense1000Hundredth|ScaleVec10000Inc20)
gonum_path Benchmark(AStarUndirectedmallWorld_10_2_2_2_Heur|Dominators/nested_if_n256)
gonum_topo Benchmark(TarjanSCCGnp_1000_half|TarjanSCCGnp_10_tenth)
gonum_traverse BenchmarkWalkAllBreadthFirstGnp_(10|1000)_tenth
gtank_blake2s BenchmarkHash8K
gtank_ed25519 Benchmark(IsOnCurve|ScalarMult)
hugo_helpers Benchmark(StripHTML|ReaderContains)
hugo_hugolib BenchmarkParsePage
hugo_hugolib_sitebuilding BenchmarkSiteBuilding/YAML,num_pages=10,num_tags=10,tags_per_page=20,shortcodes,render-12
k8s_api BenchmarkEncodeCodecFromInternalProtobuf
k8s_schedulercache BenchmarkList1kNodes30kPods
minio BenchmarkGetObject5MbFS
nelsam_gxui_interval Benchmark
semver BenchmarkValidateVersionTildeFail
spexs2 BenchmarkRun/10k/1
uber_zap BenchmarkAddingFields/(Zap.Sugar|^[ais])
uuid Benchmark(NewV5|MarshalToString)

There is a benchmark runner that automates downloading, building, and running these benchmarks under various (user-defined) configurations. Benchmarking noise on Linux can be somewhat reduced with perflock.

A few have been proposed but have so far failed to make the cut (for fetch, build, or noise problems):

short name notes go get path benchmark regexp contact
eolian_dsp Benchmark
ethereum_whisperv5 Benchmark
kanzi Benchmark
Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.