Permalink
77434f4 Jun 20, 2016
1011 lines (987 sloc) 24.2 KB

Performance Results

The last complete run of this benchmark setup yielded the results presented below. This report was generated on 2016-06-20 19:30:25.

All results are normalized to Java 1.8.0_91. Furthermore, we report peak performance. This means, the reported measurements are taken after warmup and compilation of the benchmark code is completed.

Overview

Fast Language Implementations

The following set of language implementations reaches the performance of Java on our set of benchmarks within a factor of 2 to 3 on average. To allow for a more detailed assessment of these fast language implementations, we exclude slower ones from the following plot.

plot of chunk fast-langs-overview

All Language Implementations

Other language implementations are not necessarily reaching performance similar to Java on our benchmarks. The following plot include all of the implementations.

plot of chunk all-langs-overview

Performance Overview Data

The following table contains the numerical representation of the results depicted above.

  Runtime Factor over Java
geomean sd min max median
Java 1.00 0.00 1.00 1.00 1.00
Crystal 1.85 2.45 0.79 8.90 1.49
SOMns 2.00 0.77 0.93 3.21 1.97
TruffleSOM 2.20 1.58 1.10 7.25 2.13
Graal.js 2.23 2.82 0.98 12.22 2.13
JRuby+Truffle 2.36 0.94 1.04 4.31 2.64
Node.js 2.89 3.33 1.14 12.25 2.54
JRuby 12.83 7.82 3.36 31.62 14.66
Rubinius 25.09 21.52 5.85 77.60 26.02
MRI 45.62 19.48 19.37 81.00 43.98

Details for all Benchmarks

The following plots show results for each of the benchmarks.

Fast Language Implementations

plot of chunk fast-langs-benchmarks

Slow Language Implementations

plot of chunk slow-langs-benchmarks

Benchmark Results

The following table contains the numerical results for all benchmarks.

    Runtime Factor over Java
VM geomean sd
Bounce Java 1.00 0.03
  Crystal 1.27 0.08
  SOMns 1.30 0.02
  TruffleSOM 1.52 0.03
  Graal.js 1.26 0.02
  JRuby+Truffle 1.41 0.06
  Node.js 12.25 0.01
  JRuby 14.51 0.08
  Rubinius 8.02 0.41
  MRI 38.35 0.01
CD Java 1.00 0.03
  Crystal 4.18 0.14
  SOMns 2.80 0.03
  TruffleSOM 7.25 0.04
  Graal.js 2.16 0.03
  JRuby+Truffle 2.56 0.11
  Node.js 2.46 0.01
  JRuby 10.58 0.04
  Rubinius 23.68 2.13
  MRI 66.24 0.16
DeltaBlue Java 1.00 0.06
  Crystal 1.55 0.33
  SOMns 3.20 0.18
  TruffleSOM 2.25 0.11
  Graal.js 2.10 0.06
  JRuby+Truffle 2.55 0.28
  Node.js 3.93 0.90
  JRuby 22.61 0.72
  Rubinius 25.46 10.01
  MRI 41.08 0.58
Havlak Java 1.00 0.03
  Crystal 6.79 0.27
  SOMns 2.14 0.08
  TruffleSOM 2.96 0.08
  Graal.js 12.22 0.09
  JRuby+Truffle 4.31 0.07
  Node.js 10.24 0.30
  JRuby 18.68 0.12
  Rubinius 49.65 2.48
  MRI 59.91 0.86
Json Java 1.00 0.02
  Crystal 1.70 0.05
  SOMns 1.42 0.02
  TruffleSOM 1.64 0.02
  Graal.js 1.60 0.02
  JRuby+Truffle 2.66 0.07
  Node.js 1.14 0.00
  JRuby 4.79 0.03
  Rubinius 77.60 0.36
  MRI 19.37 0.37
List Java 1.00 0.01
  Crystal 0.96 0.08
  SOMns 3.02 0.04
  TruffleSOM 2.72 0.04
  Graal.js 2.09 0.04
  JRuby+Truffle 2.62 0.10
  Node.js 2.25 0.00
  JRuby 3.36 0.03
  Rubinius 5.85 0.73
  MRI 46.87 0.02
Mandelbrot Java 1.00 0.00
  Crystal 1.10 0.00
  SOMns 1.41 0.00
  TruffleSOM 1.52 0.02
  Graal.js 1.21 0.00
  JRuby+Truffle 1.22 0.04
  Node.js 1.14 0.00
  JRuby 8.83 0.05
  Rubinius 12.83 6.34
  MRI 29.95 0.01
NBody Java 1.00 0.00
  Crystal 0.97 0.03
  SOMns 1.67 0.01
  TruffleSOM 1.11 0.01
  Graal.js 1.01 0.01
  JRuby+Truffle 1.04 0.07
  Node.js 1.62 0.01
  JRuby 18.66 0.07
  Rubinius 45.95 1.60
  MRI 62.50 0.04
Permute Java 1.00 0.00
  Crystal 2.37 0.01
  SOMns 2.87 0.04
  TruffleSOM 3.45 0.05
  Graal.js 3.06 0.05
  JRuby+Truffle 3.58 0.13
  Node.js 2.13 0.00
  JRuby 31.62 0.10
  Rubinius 67.08 2.44
  MRI 81.00 0.09
Queens Java 1.00 0.02
  Crystal 1.74 0.07
  SOMns 2.33 0.03
  TruffleSOM 2.92 0.04
  Graal.js 2.95 0.02
  JRuby+Truffle 3.23 0.08
  Node.js 2.62 0.00
  JRuby 20.07 0.14
  Rubinius 26.58 0.56
  MRI 35.49 0.01
Richards Java 1.00 0.01
  Crystal 0.79 0.00
  SOMns 1.78 0.00
  TruffleSOM 1.49 0.00
  Graal.js 2.91 0.02
  JRuby+Truffle 2.81 0.03
  Node.js 4.12 0.05
  JRuby 14.81 1.11
  Rubinius 38.01 0.75
  MRI 76.90 0.13
Sieve Java 1.00 0.02
  Crystal 1.41 0.07
  SOMns 1.79 0.02
  TruffleSOM 2.01 0.03
  Graal.js 3.61 0.02
  JRuby+Truffle 1.62 0.06
  Node.js 3.61 0.00
  JRuby 22.14 0.06
  Rubinius 20.11 3.02
  MRI 32.69 0.06
Storage Java 1.00 0.02
  Crystal 8.90 0.25
  SOMns 0.93 0.02
  TruffleSOM 1.10 0.02
  Graal.js 0.98 0.02
  JRuby+Truffle 2.77 0.16
  Node.js 2.03 0.00
  JRuby 9.77 0.06
  Rubinius 16.59 0.26
  MRI 32.15 0.32
Towers Java 1.00 0.02
  Crystal 1.43 0.00
  SOMns 3.21 0.04
  TruffleSOM 3.37 0.05
  Graal.js 2.79 0.04
  JRuby+Truffle 3.29 0.16
  Node.js 2.89 0.00
  JRuby 9.39 0.05
  Rubinius 26.90 1.29
  MRI 68.50 0.03