Skip to content

Latest commit

 

History

History
66 lines (42 loc) · 2.35 KB

README.md

File metadata and controls

66 lines (42 loc) · 2.35 KB
Raw

Hashtable latencies

This benchmark measures the latency of a web service that stores a large (250K item) hashtable, in multiple languages.

Garbage collected languages are of greater interest, and the Swift version is mostly used as a reference of what would be ideal.

Rules for implementations.

  • Each HTTP request will add one new item (and remove one if count > 250K)
  • Each item will be a new 1KB buffer with every byte initialized to a value.
  • Programs will listen on port 8080 and respond with status 200, body "OK"

Structure

  • All program directories are listed in programs.txt
  • Every directory should contain build.sh and run.sh
  • wrk2 must be installed and linked in $PATH as wrk2

Tests

  • Warmup: 9K req/s are sent for 60s (initial hash table content)
  • Test : 9K req/s are sent by 99 clients concurrently for 180s

Output:

  • All reports are in reports directory.
  • Charts from both tests can be generated by using hdrhistogram

Current results

chart

Program notes

  • go - Go version is go1.6.2 darwin/amd64; uses fasthttp. Using the built-in http library results with a lot more work for the GC, and a chart similar to the one seen for Haskell.
  • ocaml-reason - This is built with OCaml 4.03+flambda (ReasonML is currently not compatible with OCaml 4.03 so the ML version was generated from it using refmt first). Follow the instructions here: https://ocaml.org/docs/install.html
  • node - Requires node 6.x (Buffer.alloc API)
  • haskell - Should build easily if you have stack
  • swift-zewo - You should setup swiftenv to the right snapshot

Conclusions

Want to build low-latency, memory-intensive services in a GCed language? Use OCaml (or Reason if you prefer the syntax).

OCaml GC notes

Other things that were attempted to make the latency worse for OCaml:

  • random buffer sizes
  • longer running time

TODO:

  • simulate extra GC work on a percentage of requests (i.e. creating T temporary objects of size S)