Skip to content
benchmark of multiple different possible representations of a 2D board in elixir
Elixir
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config
lib
test
.formatter.exs
.gitignore
LICENSE
README.md
benchmark.exs
mix.exs
mix.lock

README.md

ElixirBoardsBenchmark

A benchmark of different possible implementations of a 2D game board in elixir. The current benchmark is specifically for the board size 9x9.

What is a board implementation? See the Board behaviour - creating a new board, get(x, y) and set(x, y).

What is benchmarked?

  • get of (0, 0), (4, 4) and (8, 8)
  • set of (0, 0), (4, 4) and (8, 8)
  • getting and setting of (0, 0), (4, 4) and (8, 8)
  • getting and setting of all coordinates in one rune

The benchmarking code itself is in benchmark.exs.

Table of Contents

But Why?

Fun. No concrete use case at the moment other than looking at performance of different data structures in these circumstances. Additionally, this happened to have been one of the first questions I asked the elixir community.

Adding your own implemenation?

Have another implementation? Great! Add it to lib/board/ with a name describing the data structure, adopt the Board behaviour then add the module name to the module list in test/board_test.exs and benchmark.exs. Done! (if the tests pass and the benchmark runs of course)

Results

System Information

Operating System Linux
CPU Information Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
Number of Available Cores 8
Available Memory 15.61 GB
Elixir Version 1.8.2
Erlang Version 22.0

Getting and Setting Full Board

Run Time

Name IPS Average Devitation Median 99th %
Tuple1D 133.95 K 7.47 μs ±23.29% 6.93 μs 10.80 μs
Tuple2D 132.16 K 7.57 μs ±29.17% 7.21 μs 14.02 μs
MapTuple 126.54 K 7.90 μs ±25.69% 7.59 μs 16.30 μs
ProcessDictionary 64.68 K 15.46 μs ±14.61% 15.12 μs 21.71 μs
ETSSet 60.35 K 16.57 μs ±9.17% 16.04 μs 21.09 μs
Array2D 56.76 K 17.62 μs ±17.45% 17.15 μs 21.45 μs
MapTupleFull 55.44 K 18.04 μs ±11.00% 16.92 μs 22.01 μs
MapTupleHalfFull 53.70 K 18.62 μs ±8.36% 17.96 μs 22.32 μs
Array1D 50.74 K 19.71 μs ±10.60% 19.29 μs 22.71 μs
ETSOrderedSet 39.53 K 25.30 μs ±10.51% 24.82 μs 28.42 μs
Map2D 36.24 K 27.59 μs ±8.32% 27.71 μs 32.27 μs
List2D 29.65 K 33.73 μs ±4.12% 33.31 μs 36.99 μs
MapTupleQuarterFull 28.23 K 35.42 μs ±3.86% 34.96 μs 38.34 μs
List1D 15.41 K 64.90 μs ±2.84% 64.91 μs 70.59 μs

Comparison

Name IPS Slower
Tuple1D 133.95 K  
Tuple2D 132.16 K 1.01x
MapTuple 126.54 K 1.06x
ProcessDictionary 64.68 K 2.07x
ETSSet 60.35 K 2.22x
Array2D 56.76 K 2.36x
MapTupleFull 55.44 K 2.42x
MapTupleHalfFull 53.70 K 2.49x
Array1D 50.74 K 2.64x
ETSOrderedSet 39.53 K 3.39x
Map2D 36.24 K 3.7x
List2D 29.65 K 4.52x
MapTupleQuarterFull 28.23 K 4.75x
List1D 15.41 K 8.69x

Memory Usage

Name Memory Factor
Tuple1D 54.91 KB  
Tuple2D 18.51 KB 0.34x
MapTuple 12.74 KB 0.23x
ProcessDictionary 9.52 KB 0.17x
ETSSet 11.55 KB 0.21x
Array2D 34.33 KB 0.63x
MapTupleFull 25.79 KB 0.47x
MapTupleHalfFull 24.06 KB 0.44x
Array1D 38.10 KB 0.69x
ETSOrderedSet 11.55 KB 0.21x
Map2D 49.55 KB 0.9x
List2D 22.60 KB 0.41x
MapTupleQuarterFull 47.56 KB 0.87x
List1D 59.38 KB 1.08x

Mixed Bag (3 sets, 3 gets)

Run Time

Name IPS Average Devitation Median 99th %
Tuple1D 6.02 M 166.04 ns ±721.06% 127 ns 424 ns
Tuple2D 5.89 M 169.76 ns ±1323.53% 153 ns 350 ns
MapTuple 5.01 M 199.45 ns ±907.01% 178 ns 296 ns
ProcessDictionary 3.09 M 323.35 ns ±379.04% 307 ns 547 ns
Array1D 2.16 M 462.20 ns ±835.43% 421 ns 698 ns
MapTupleFull 1.84 M 542.80 ns ±29.17% 506 ns 730 ns
ETSSet 1.83 M 547.30 ns ±32.92% 515 ns 1474.88 ns
MapTupleHalfFull 1.79 M 560.05 ns ±41.55% 528 ns 953.76 ns
Array2D 1.76 M 569.76 ns ±405.50% 528 ns 873 ns
Map2D 1.29 M 777.13 ns ±19.67% 754 ns 2011.91 ns
ETSOrderedSet 1.10 M 907.35 ns ±39.78% 853 ns 1961.88 ns
List2D 0.91 M 1093.69 ns ±30.31% 1068 ns 1514.97 ns
MapTupleQuarterFull 0.82 M 1215.76 ns ±36.83% 1183 ns 2882 ns
List1D 0.44 M 2277.54 ns ±171.70% 2160 ns 3321.13 ns

Comparison

Name IPS Slower
Tuple1D 6.02 M  
Tuple2D 5.89 M 1.02x
MapTuple 5.01 M 1.2x
ProcessDictionary 3.09 M 1.95x
Array1D 2.16 M 2.78x
MapTupleFull 1.84 M 3.27x
ETSSet 1.83 M 3.3x
MapTupleHalfFull 1.79 M 3.37x
Array2D 1.76 M 3.43x
Map2D 1.29 M 4.68x
ETSOrderedSet 1.10 M 5.46x
List2D 0.91 M 6.59x
MapTupleQuarterFull 0.82 M 7.32x
List1D 0.44 M 13.72x

Memory Usage

Name Memory Factor
Tuple1D 1344 B  
Tuple2D 512 B 0.38x
MapTuple 368 B 0.27x
ProcessDictionary 104 B 0.08x
Array1D 1088 B 0.81x
MapTupleFull 400 B 0.3x
ETSSet 248 B 0.18x
MapTupleHalfFull 384 B 0.29x
Array2D 1112 B 0.83x
Map2D 1712 B 1.27x
ETSOrderedSet 248 B 0.18x
List2D 656 B 0.49x
MapTupleQuarterFull 1624 B 1.21x
List1D 2048 B 1.52x

get(0,0)

Run Time

Name IPS Average Devitation Median 99th %
Tuple1D 44.12 M 22.66 ns ±842.77% 20 ns 33 ns
Tuple2D 42.46 M 23.55 ns ±846.67% 20 ns 64 ns
Array1D 30.38 M 32.92 ns ±84.61% 32 ns 49 ns
MapTuple 29.09 M 34.38 ns ±111.15% 32 ns 61 ns
MapTupleQuarterFull 18.86 M 53.03 ns ±37.27% 50 ns 95 ns
Array2D 18.62 M 53.70 ns ±67.02% 50 ns 80 ns
List1D 18.26 M 54.75 ns ±56.06% 53 ns 117 ns
ProcessDictionary 17.19 M 58.18 ns ±1393.09% 52 ns 71 ns
Map2D 15.79 M 63.34 ns ±25.86% 60 ns 108 ns
MapTupleHalfFull 10.54 M 94.87 ns ±27.72% 91 ns 142 ns
MapTupleFull 10.29 M 97.16 ns ±18.01% 93 ns 132 ns
ETSSet 9.74 M 102.63 ns ±26.57% 100 ns 144 ns
List2D 9.04 M 110.57 ns ±69.64% 105 ns 200 ns
ETSOrderedSet 6.47 M 154.65 ns ±19.27% 152 ns 190 ns

Comparison

Name IPS Slower
Tuple1D 44.12 M  
Tuple2D 42.46 M 1.04x
Array1D 30.38 M 1.45x
MapTuple 29.09 M 1.52x
MapTupleQuarterFull 18.86 M 2.34x
Array2D 18.62 M 2.37x
List1D 18.26 M 2.42x
ProcessDictionary 17.19 M 2.57x
Map2D 15.79 M 2.79x
MapTupleHalfFull 10.54 M 4.19x
MapTupleFull 10.29 M 4.29x
ETSSet 9.74 M 4.53x
List2D 9.04 M 4.88x
ETSOrderedSet 6.47 M 6.82x

get(4, 4)

Run Time

Name IPS Average Devitation Median 99th %
Tuple2D 37.76 M 26.48 ns ±722.24% 24 ns 40 ns
Tuple1D 36.87 M 27.12 ns ±618.93% 24 ns 57 ns
Array1D 28.57 M 35.00 ns ±78.21% 34 ns 45 ns
MapTuple 27.00 M 37.04 ns ±131.99% 34 ns 73 ns
ProcessDictionary 18.89 M 52.94 ns ±843.86% 49 ns 67 ns
Array2D 16.70 M 59.88 ns ±34.66% 56 ns 85 ns
Map2D 14.28 M 70.03 ns ±49.51% 64 ns 115 ns
MapTupleHalfFull 10.38 M 96.36 ns ±30.87% 93 ns 146 ns
MapTupleFull 9.97 M 100.30 ns ±20.77% 97 ns 136 ns
ETSSet 9.17 M 109.02 ns ±14.73% 107 ns 145 ns
List2D 5.77 M 173.18 ns ±47.97% 164 ns 293 ns
ETSOrderedSet 5.64 M 177.34 ns ±34.69% 173 ns 211 ns
MapTupleQuarterFull 3.99 M 250.61 ns ±25.89% 247 ns 308 ns
List1D 2.46 M 407.10 ns ±14.32% 400 ns 632 ns

Comparison

Name IPS Slower
Tuple2D 37.76 M  
Tuple1D 36.87 M 1.02x
Array1D 28.57 M 1.32x
MapTuple 27.00 M 1.4x
ProcessDictionary 18.89 M 2.0x
Array2D 16.70 M 2.26x
Map2D 14.28 M 2.64x
MapTupleHalfFull 10.38 M 3.64x
MapTupleFull 9.97 M 3.79x
ETSSet 9.17 M 4.12x
List2D 5.77 M 6.54x
ETSOrderedSet 5.64 M 6.7x
MapTupleQuarterFull 3.99 M 9.46x
List1D 2.46 M 15.37x

get (8, 8)

Run Time

Name IPS Average Devitation Median 99th %
Tuple2D 42.47 M 23.55 ns ±788.60% 21 ns 40 ns
Tuple1D 40.98 M 24.40 ns ±725.07% 22 ns 41 ns
Array1D 29.67 M 33.70 ns ±161.51% 33 ns 45 ns
MapTuple 28.54 M 35.03 ns ±986.95% 32 ns 50 ns
ProcessDictionary 19.71 M 50.73 ns ±1279.45% 47 ns 65 ns
Array2D 17.88 M 55.92 ns ±85.10% 52 ns 80 ns
Map2D 13.28 M 75.31 ns ±32.34% 73 ns 126 ns
MapTupleHalfFull 12.12 M 82.53 ns ±31.49% 80 ns 126 ns
ETSSet 9.90 M 101.05 ns ±16.04% 99 ns 135 ns
MapTupleFull 9.85 M 101.53 ns ±19.29% 99 ns 136 ns
ETSOrderedSet 5.59 M 178.80 ns ±41.70% 169 ns 394.49 ns
MapTupleQuarterFull 4.09 M 244.65 ns ±16.85% 242 ns 299 ns
List2D 3.76 M 265.82 ns ±35.71% 251 ns 486 ns
List1D 1.38 M 724.35 ns ±10.88% 715 ns 951 ns

Comparison

Name IPS Slower
Tuple2D 42.47 M  
Tuple1D 40.98 M 1.04x
Array1D 29.67 M 1.43x
MapTuple 28.54 M 1.49x
ProcessDictionary 19.71 M 2.15x
Array2D 17.88 M 2.37x
Map2D 13.28 M 3.2x
MapTupleHalfFull 12.12 M 3.5x
ETSSet 9.90 M 4.29x
MapTupleFull 9.85 M 4.31x
ETSOrderedSet 5.59 M 7.59x
MapTupleQuarterFull 4.09 M 10.39x
List2D 3.76 M 11.29x
List1D 1.38 M 30.76x

set(0, 0)

Run Time

Name IPS Average Devitation Median 99th %
List1D 31.04 M 32.21 ns ±59.43% 30 ns 91 ns
Tuple2D 16.91 M 59.15 ns ±2301.62% 48 ns 87 ns
ProcessDictionary 14.86 M 67.31 ns ±666.24% 62 ns 80 ns
MapTuple 14.65 M 68.24 ns ±24850.94% 33 ns 88 ns
Tuple1D 12.04 M 83.05 ns ±5738.15% 43 ns 116 ns
List2D 10.49 M 95.30 ns ±484.89% 81 ns 204 ns
MapTupleQuarterFull 9.01 M 111.00 ns ±247.51% 81 ns 1388 ns
ETSSet 8.75 M 114.30 ns ±19.08% 112 ns 142 ns
MapTupleFull 8.65 M 115.64 ns ±16.84% 111 ns 151 ns
MapTupleHalfFull 6.87 M 145.65 ns ±195.37% 121 ns 174.11 ns
Array2D 6.73 M 148.59 ns ±31.02% 145 ns 237 ns
Array1D 5.77 M 173.25 ns ±1013.39% 149 ns 189 ns
ETSOrderedSet 5.49 M 182.25 ns ±22.92% 180 ns 227.75 ns
Map2D 4.09 M 244.47 ns ±115.24% 219 ns 1051 ns

Comparison

Name IPS Slower
List1D 31.04 M  
Tuple2D 16.91 M 1.84x
ProcessDictionary 14.86 M 2.09x
MapTuple 14.65 M 2.12x
Tuple1D 12.04 M 2.58x
List2D 10.49 M 2.96x
MapTupleQuarterFull 9.01 M 3.45x
ETSSet 8.75 M 3.55x
MapTupleFull 8.65 M 3.59x
MapTupleHalfFull 6.87 M 4.52x
Array2D 6.73 M 4.61x
Array1D 5.77 M 5.38x
ETSOrderedSet 5.49 M 5.66x
Map2D 4.09 M 7.59x

set(4, 4)

Run Time

Name IPS Average Devitation Median 99th %
MapTuple 21.17 M 47.25 ns ±18021.59% 24 ns 55 ns
Tuple2D 20.96 M 47.70 ns ±4316.38% 38 ns 57 ns
ProcessDictionary 17.43 M 57.36 ns ±6745.68% 46 ns 62 ns
Tuple1D 12.43 M 80.46 ns ±8459.34% 33 ns 115 ns
MapTupleFull 9.54 M 104.85 ns ±25.06% 100 ns 242.92 ns
ETSSet 9.34 M 107.08 ns ±20.13% 105 ns 137 ns
MapTupleHalfFull 8.92 M 112.14 ns ±51.21% 108 ns 157 ns
Array2D 7.06 M 141.62 ns ±45.86% 138 ns 211 ns
Array1D 6.01 M 166.46 ns ±949.89% 142 ns 186 ns
ETSOrderedSet 5.55 M 180.21 ns ±30.79% 174 ns 271.61 ns
List2D 4.81 M 207.77 ns ±93.27% 201 ns 431 ns
MapTupleQuarterFull 4.03 M 247.92 ns ±28.34% 241 ns 307 ns
Map2D 3.95 M 253.35 ns ±117.41% 225 ns 1053.34 ns
List1D 2.80 M 357.35 ns ±94.13% 342 ns 611 ns

Comparison

Name IPS Slower
MapTuple 21.17 M  
Tuple2D 20.96 M 1.01x
ProcessDictionary 17.43 M 1.21x
Tuple1D 12.43 M 1.7x
MapTupleFull 9.54 M 2.22x
ETSSet 9.34 M 2.27x
MapTupleHalfFull 8.92 M 2.37x
Array2D 7.06 M 3.0x
Array1D 6.01 M 3.52x
ETSOrderedSet 5.55 M 3.81x
List2D 4.81 M 4.4x
MapTupleQuarterFull 4.03 M 5.25x
Map2D 3.95 M 5.36x
List1D 2.80 M 7.56x

set(8, 8)

Run Time

Name IPS Average Devitation Median 99th %
Tuple2D 16.87 M 59.28 ns ±2026.15% 51 ns 70 ns
ProcessDictionary 14.36 M 69.65 ns ±4696.55% 59 ns 75 ns
MapTuple 12.16 M 82.24 ns ±23284.12% 36 ns 67 ns
Tuple1D 10.67 M 93.74 ns ±7351.11% 46 ns 111 ns
MapTupleHalfFull 8.92 M 112.16 ns ±42.16% 110 ns 158 ns
MapTupleFull 8.77 M 114.03 ns ±22.42% 111 ns 148 ns
ETSSet 8.58 M 116.53 ns ±17.78% 114 ns 145 ns
Array2D 6.58 M 152.06 ns ±35.69% 148 ns 245 ns
Array1D 5.63 M 177.74 ns ±978.38% 154 ns 202 ns
ETSOrderedSet 5.34 M 187.42 ns ±38.95% 183 ns 228 ns
Map2D 3.50 M 285.55 ns ±100.73% 260 ns 1090 ns
List2D 2.89 M 345.55 ns ±24.43% 328 ns 587.01 ns
MapTupleQuarterFull 2.62 M 381.31 ns ±16.51% 375 ns 436 ns
List1D 1.35 M 739.31 ns ±147.87% 705 ns 1340.80 ns

Comparison

Name IPS Slower
Tuple2D 16.87 M  
ProcessDictionary 14.36 M 1.17x
MapTuple 12.16 M 1.39x
Tuple1D 10.67 M 1.58x
MapTupleHalfFull 8.92 M 1.89x
MapTupleFull 8.77 M 1.92x
ETSSet 8.58 M 1.97x
Array2D 6.58 M 2.57x
Array1D 5.63 M 3.0x
ETSOrderedSet 5.34 M 3.16x
Map2D 3.50 M 4.82x
List2D 2.89 M 5.83x
MapTupleQuarterFull 2.62 M 6.43x
List1D 1.35 M 12.47x

You can’t perform that action at this time.