Renaissance Benchmark Suite
The Renaissance Benchmark Suite is an open source collaborative benchmark project where the community can propose and improve benchmark workloads. This repository serves to track and process measurement results for the suite. Included here are the performance measurement summaries, for full data see the Renaissance Benchmark Results Repository.
Please note that performance measurements depend on many complex factors and therefore YMMV. The results given here serve only as examples, you should always collect your own measurements. (But do let us know if your results are significantly different from the results given here so that we can look.)
The current measurements were collected with Renaissance
91cf51d on multiple
8 core Intel Xeon E5-2620 v4 machines at 2100 GHz with 64 GB RAM,
running bare metal Fedora Linux 31 with kernel 5.3.8.
Each benchmark was run multiple times, each run was executed in a new JVM instance and terminated after 10 minutes. The durations of all repetitions were collected and are included in the data, but only the second half of each run is used in the plots as warm data.
As a shared setting, the JVM implementations were executed with the
-Xms12G -Xmx12G command line options,
which serve to fix the heap size and reduce variability due to heap sizing.
Except as noted below, other settings were left at default values.
The JVM implementations referenced in the results are:
OpenJDK is the GraalVM Community Edition JVM implementation run with
-XX:-EnableJVMCI -XX:-UseJVMCICompilerto force the use of the default OpenJDK JIT compiler.
GraalVM CE is GraalVM Community Edition 21.2.0-dev.
> java -version openjdk version "1.8.0_302" OpenJDK Runtime Environment (build 1.8.0_302-b07) OpenJDK 64-Bit Server VM GraalVM CE 21.2.0-dev (build 25.302-b07-jvmci-21.2-b06, mixed mode)
> java -version openjdk version "11.0.12" 2021-07-20 OpenJDK Runtime Environment GraalVM CE 21.2.0-dev (build 11.0.12+6-jvmci-21.2-b06) OpenJDK 64-Bit Server VM GraalVM CE 21.2.0-dev (build 11.0.12+6-jvmci-21.2-b06, mixed mode, sharing)
- GraalVM EE is GraalVM Enterprise Edition 21.2.0-dev.
> java -version java version "1.8.0_301" Java(TM) SE Runtime Environment (build 1.8.0_301-b09) Java HotSpot(TM) 64-Bit Server VM GraalVM EE 21.2.0-dev (build 25.301-b09-jvmci-21.2-b06, mixed mode)
> java -version java version "11.0.12" 2021-07-20 LTS Java(TM) SE Runtime Environment GraalVM EE 21.2.0-dev (build 11.0.12+8-LTS-jvmci-21.2-b06) Java HotSpot(TM) 64-Bit Server VM GraalVM EE 21.2.0-dev (build 11.0.12+8-LTS-jvmci-21.2-b06, mixed mode, sharing)
Mean Repetition Times
The figure shows the mean repetition time for each benchmark, computed as the average duration of all warm repetitions. The error bars show 99% confidence intervals for the mean computed using bootstrap.
Individual Repetition Times
The figure shows the individual repetition times for each benchmark in a violin plot. The violin shape is the widest at the height of the most frequent repetition times, the box inside the shape stretches from the low to the high quartile, with a mark at the median. Floating window outlier filtering was used to discard no more than 10% of most extreme observations, to preserve plot scale.