Microbenchmarking tool for Elixir
Branch: master
Clone or download
Latest commit d4546fe Aug 2, 2017
Type Name Latest commit message Commit time
Failed to load latest commit information.
bench Fix execution warning from deprecated Erlang :random by converting to… Aug 1, 2017
lib Merge branch 'alco/elixir-1.5-deprecations' Aug 2, 2017
priv Embed styles and javascript into index.html Jul 21, 2014
test Basic chart Jul 16, 2014
.gitignore Ignore the snapshots directory Apr 1, 2015
README.md Replace :random with :rand in the Readme Aug 1, 2017
bench_graph.png Update Readme, add License Jul 20, 2014
mix.exs Bump to 0.3.5 Aug 2, 2017
mix.lock Update ex_doc to fix Elixir v1.5 deprecations Aug 2, 2017



Hex version Hex downloads

Microbenchmarking tool for Elixir.


With Benchfella you can define small tests and it will intelligently run each individual one to obtain a more or less reliable estimate of the average run time for each test.

The key features of Benchfella:

  • very easy to write and run microbenchmarks
  • compare measurements between multiple runs
  • plot results of multiple runs

If you are looking for a more elaborate treatment of the measurements, take a look at bmark which employs mathematical statistics to compare benchmarking results and determine their credibility.


Add benchfella as a dependency to your project:

# in your mix.exs

defp deps do
  [{:benchfella, "~> 0.3.0"}]

This will make the new tasks available in the root directory of your Mix project.

Any project will do, so if you just want to measure a snippet of code quickly, create a bare-bones Mix project with mix new, create a subdirectory called bench in it and put your benchmark definitions there. See examples below.


Take a moment to study the output of running mix help bench and mix help bench.cmp inside your Mix project to see all supported options.

In order to start writing tests, add a directory called bench and put files with names that match the pattern *_bench.exs in it. Then run mix bench in the root directory of your project. Benchfella will then load each test and execute it for as many iterations as necessary so that the total running time is at least the specified duration.


# bench/basic_bench.exs
defmodule BasicBench do
  use Benchfella

  @list Enum.to_list(1..1000)

  bench "hello list" do
    Enum.reverse @list
$ mix bench
  duration:      1.0 s

## BasicBench
[13:23:58] 0/1: hello list

Finished in 3.15 seconds

## BasicBench
hello list      500000   5.14 µs/op

In the end, the number of iterations and the average time of a single iteration are printed to the standard output. Additionally, the output in machine format is written to a snapshot file in bench/snapshots/.

setup_all and teardown_all

setup_all/0 lets you perform some code before the first test in a module is run. It takes no arguments and should return {:ok, <context>} where <context> is any term, it will be passed into before_each_bench/1 and teardown_all/1 if they are defined. Returning any other value will raise an error and cause the whole module to be skipped.

teardown_all/1 lets you do some cleanup after the last test in a module has finished running. It takes the context returned from setup_all/0 (nil by default) as its argument.

# bench/sys_bench.exs
defmodule SysBench do
  use Benchfella

  setup_all do
    depth = :erlang.system_flag(:backtrace_depth, 100)
    {:ok, depth}

  teardown_all depth do
    :erlang.system_flag(:backtrace_depth, depth)

  @list Enum.to_list(1..10000)

  bench "list reverse" do

before_each_bench and after_each_bench

before_each_bench/1 runs before each individual test is executed. It takes the context returned from setup_all/0 and should return {:ok, <bench_context>} where <bench_context> is any term. Returning any other value will raise an error and cause the current test to be skipped.

<bench_context> returned from before_each_bench/1 will be available as the bench_context variable in each test.

after_each_bench/1 runs after each individual test has been executed. It takes the context returned from before_each_bench/1 as its argument.

# bench/ets_bench.exs
defmodule ETSBench do
  use Benchfella

  before_each_bench _ do
    tid = :ets.new(:my_table, [:public])
    {:ok, tid}

  after_each_bench tid do
    IO.inspect length(:ets.tab2list(tid))

  bench "ets insert", [_unused: inspect_table(bench_context)] do
    tid = bench_context
    :ets.insert(tid, {:rand.uniform(1000), :x})

  defp inspect_table(tid) do
    IO.inspect :ets.info(tid)

Run time values

When you need to generate inputs for tests at run time without affecting the measurements and you can't use before_each_bench/1 hook for that, the following trick can be used:

# bench/string_bench.exs
defmodule StringBench do
  use Benchfella

  bench "reverse string", [str: gen_string()] do

  defp gen_string() do
    String.duplicate("abc", 10000)

mix bench.cmp

To compare results between multiple runs, use mix bench.cmp.

# Run 'mix bench' one more time.
# Each run automatically saves a snapshot in bench/snapshots.
$ mix bench

# 'mix bench.cmp' will read the two latest snapshots by default.
# You could also pass the snapshot files to compare as arguments.
$ mix bench.cmp -d percent
bench/snapshots/2015-03-26_01-17-17.snapshot vs

## ListBench
reverse list              -10.32%

## StringBench
reverse string dynamic    +2.26%
reverse string            +3.33%

mix bench.graph

Benchfella can produce an HTML page with graphs providing various insights into the raw data obtained from running mix bench.

# run the benchmarks twice
$ mix bench
$ mix bench

# 'mix bench.graph' works similarly to 'mix bench.cmp' except it can display
# all given snapshots on one graph.
$ mix bench.graph
Wrote bench/graphs/index.html

$ open bench/graphs/index.html

Graph example


This software is licensed under the MIT license.