Skip to content
Latency of a web service that stores a large hashtable, in multiple languages
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
go
haskell
node
ocaml-reason
python-uwsgi
reports
swift-zewo
README.md
programs.txt
test.sh

README.md

Hashtable latencies

This benchmark measures the latency of a web service that stores a large (250K item) hashtable, in multiple languages.

Garbage collected languages are of greater interest, and the Swift version is mostly used as a reference of what would be ideal.

Rules for implementations.

  • Each HTTP request will add one new item (and remove one if count > 250K)
  • Each item will be a new 1KB buffer with every byte initialized to a value.
  • Programs will listen on port 8080 and respond with status 200, body "OK"

Structure

  • All program directories are listed in programs.txt
  • Every directory should contain build.sh and run.sh
  • wrk2 must be installed and linked in $PATH as wrk2

Tests

  • Warmup: 9K req/s are sent for 60s (initial hash table content)
  • Test : 9K req/s are sent by 99 clients concurrently for 180s

Output:

  • All reports are in reports directory.
  • Charts from both tests can be generated by using hdrhistogram

Current results

chart

Program notes

  • go - Go version is go1.6.2 darwin/amd64; uses fasthttp. Using the built-in http library results with a lot more work for the GC, and a chart similar to the one seen for Haskell.
  • ocaml-reason - This is built with OCaml 4.03+flambda (ReasonML is currently not compatible with OCaml 4.03 so the ML version was generated from it using refmt first). Follow the instructions here: https://ocaml.org/docs/install.html
  • node - Requires node 6.x (Buffer.alloc API)
  • haskell - Should build easily if you have stack
  • swift-zewo - You should setup swiftenv to the right snapshot

Conclusions

Want to build low-latency, memory-intensive services in a GCed language? Use OCaml (or Reason if you prefer the syntax).

OCaml GC notes

Other things that were attempted to make the latency worse for OCaml:

  • random buffer sizes
  • longer running time

TODO:

  • simulate extra GC work on a percentage of requests (i.e. creating T temporary objects of size S)
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.