Skip to content
Permalink
Browse files
Adds Java 17 support. (#169)
* Removes some untested JVM options leftover from Redis benchmarking.
* Adds per-role heap and GC project configurations.
* Extends maxKey option to all benchmarks.
* Uses Gradle Toolchain feature.
  • Loading branch information
pivotal-jbarrett committed May 26, 2022
1 parent 4ffee41 commit fb25358479d1e0b90b326289eda14f743a8e8cb4
Showing 32 changed files with 1,162 additions and 141 deletions.
@@ -50,59 +50,61 @@ For example:
The benchmarks can take configuration options. Some using Gradle's `-P` flag and other, which adjust
benchmark behavior, via Java system properties using `-D`.

| Option | Description |
| --------------------- | ----------- |
| `-Phosts` | Hosts used by benchmarks on the order of client,locator,server,server (-Phosts=localhost,localhost,localhost,localhost) |
| `-PoutputDir` | Results output directory (-PoutputDir=/tmp/results) |
| `-PtestJVM` | Path to an alternative JVM for running the client, locator, and servers. If not specified JAVA_HOME will be used. Note all compilation tasks will still use JAVA_HOME. |
| `-Pbenchmark.X` | Where X is a benchmark configuration, defined below. |
| `--tests` | Specific benchmarks to run (--tests=PartitionedPutBenchmark) |
| `-d` | Debug |
| `-i` | Info |
| Option | Description |
|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `-Phosts` | Hosts used by benchmarks on the order of client,locator,server,server (-Phosts=localhost,localhost,localhost,localhost) |
| `-PoutputDir` | Results output directory (-PoutputDir=/tmp/results) |
| `-Pbenchmark.X` | Where X is a benchmark configuration, defined below. |
| `-Pbenchmark.R.X` | Applies properties to specific roles. Where R is the role, "client", "server", "locator", X is a benchmark configuration, defined below. |
| `--tests` | Specific benchmarks to run (--tests=PartitionedPutBenchmark) |
| `-d` | Debug |
| `-i` | Info |
| ~~`-PtestJVM`~~ | (Deprecated, use `-Pbenchmark.withToolchainVersion`) ~~Path to an alternative JVM for running the client, locator, and servers. If not specified JAVA_HOME will be used. Note all compilation tasks will still use JAVA_HOME.~~ |

#### Benchmark Configuration
##### Common
These options may apply to all benchmarks.

| Option | Description |
| --------------------- | ----------- |
| withGc | Select which GC to use. Valid values CMS (default), G1, Z, Shenandoah, Epsilon. |
| withHeap | Specify how large a heap the benchmark VMs should use, default "8g". Accepts any `-Xmx` value, like "32g". |
| withThreads | Specify how many threads to use when executing the benchmark. Default varies by benchmark. |
| withWarmup | Specify how long to warm up the benchmark in seconds. Default is 60 seconds. |
| withDuration | Specify how long to run the benchmark in seconds. Default is 300 seconds. |
| withMinKey | The minimum key value in the key range. Default is 0. |
| withMaxKey | The maximum key value in the key range. Default varies by benchmark. |
| withLocatorCount | Number of locators a topology should use. Typically defaults to 1. |
| withServerCount | Number of servers a topology should use. Typically defaults to 2. |
| withClientCount | Number of clients a topology should use. Typically defaults to 1. |
| withReplicas | Number of region replicas. |
| withAsyncReplication | Enable asynch region replication. |
| withNettyThreads | Number of threads Netty IO Services should have. |
| Option | Description |
|----------------------|----------------------------------------------------------------------------------------------------------------------------------|
| withToolchainVersion | Version number of alternative JVM for running benchmarks, 8 (default), 11, 17. |
| withGc | Select which GC to use. Valid values CMS (default), G1, Z, Shenandoah, Epsilon. (optionally per-role) |
| withHeap | Specify how large a heap the benchmark VMs should use, default "8g". Accepts any `-Xmx` value, like "32g". (optionally per-role) |
| withThreads | Specify how many threads to use when executing the benchmark. Default varies by benchmark. |
| withWarmup | Specify how long to warm up the benchmark in seconds. Default is 60 seconds. |
| withDuration | Specify how long to run the benchmark in seconds. Default is 300 seconds. |
| withMinKey | The minimum key value in the key range. Default is 0. |
| withMaxKey | The maximum key value in the key range. Default varies by benchmark. |
| withLocatorCount | Number of locators a topology should use. Typically defaults to 1. |
| withServerCount | Number of servers a topology should use. Typically defaults to 2. |
| withClientCount | Number of clients a topology should use. Typically defaults to 1. |
| withReplicas | Number of region replicas. |
| withAsyncReplication | Enable asynch region replication. |
| withNettyThreads | Number of threads Netty IO Services should have. |

##### Geode Benchmarks
These options only apply to Geode benchmarks.

| Option | Description |
| --------------------- | ----------- |
| withSsl | Flag to run geode with SSL. A self-signed certificate will be generated at runtime. |
| withSslProtocols | Specifies enabled SSL protocols. See Geode property `ssl-protocols` |
| withSslCiphers | Specifies enabled SSL chipher suites. See Geode property `ssl-ciphers` |
| withSecurityManager | Flag to start Geode with the example implementation of SecurityManager |
| withSniProxy | Use SNI proxy topology. |
| withSniProxyImage | Provide an alternative Docker image coordinate for SNI proxy. |
| withRouter | Use router with SNI proxy topology. |
| withRouterImage | Provide an alternative Docker image coordinate for router. |
| Option | Description |
|---------------------|-------------------------------------------------------------------------------------|
| withSsl | Flag to run geode with SSL. A self-signed certificate will be generated at runtime. |
| withSslProtocols | Specifies enabled SSL protocols. See Geode property `ssl-protocols` |
| withSslCiphers | Specifies enabled SSL chipher suites. See Geode property `ssl-ciphers` |
| withSecurityManager | Flag to start Geode with the example implementation of SecurityManager |
| withSniProxy | Use SNI proxy topology. |
| withSniProxyImage | Provide an alternative Docker image coordinate for SNI proxy. |
| withRouter | Use router with SNI proxy topology. |
| withRouterImage | Provide an alternative Docker image coordinate for router. |

##### Debugging
These options should not be used when measuring benchmarks.

| Option | Description |
| --------------------- | ----------- |
| withValidation | Enable validation of operations. Default disabled.|
| withGcLogging | Enable GC logging. Default disabled.|
| withSafepointLogging | Enable Safepoint logging. Default disabled.|
| withStrace | Launch remote JVM via strace for tracing system calls. Default disabled.|
| Option | Description |
|----------------------|--------------------------------------------------------------------------|
| withValidation | Enable validation of operations. Default disabled. |
| withGcLogging | Enable GC logging. Default disabled. |
| withSafepointLogging | Enable Safepoint logging. Default disabled. |
| withStrace | Launch remote JVM via strace for tracing system calls. Default disabled. |


### Scripts for running in aws and analyzing results
@@ -101,7 +101,12 @@ test{
}

task benchmark(type: Test) {
if (project.hasProperty("benchmark.withToolchainVersion"))
javaLauncher = javaToolchains.launcherFor {
languageVersion = JavaLanguageVersion.of(project.findProperty("benchmark.withToolchainVersion"))
}
if (project.hasProperty('testJVM') && !testJVM.trim().isEmpty()) {
logger.warn("-PtestJVM is deprecated. Use -Pbenchmark.withToolchainVersion.")
executable = "${testJVM}/bin/java"
}
outputs.upToDateWhen { false }
@@ -119,6 +124,14 @@ task benchmark(type: Test) {
forkEvery 1
failFast = true


def resolvedJavaLauncher = javaLauncher.getOrNull()
if (resolvedJavaLauncher != null) {
if (resolvedJavaLauncher.getMetadata().languageVersion >= JavaLanguageVersion.of(17)) {
jvmArgs += "@${project.projectDir}/src/main/resources/open-all-jdk-packages-linux-openjdk-17"
}
}

systemProperty 'org.slf4j.simpleLogger.showDateTime', 'true'
systemProperty 'org.slf4j.simpleLogger.dateTimeFormat', 'yyyy-MM-dd HH:mm:ss.SSS'
systemProperty 'org.slf4j.simpleLogger.showThreadName', 'false'
@@ -141,41 +154,30 @@ task benchmark(type: Test) {
//prefix convention. They will be passed on to the JVM for now to not break
//CI or peoples scripts. Remove these soon!
//------------------------------------------------------------
if (project.hasProperty('withGc')) {
systemProperty 'benchmark.withGc', project.findProperty('withGc')
}
if (project.hasProperty('withHeap')) {
systemProperty 'benchmark.withHeap', project.findProperty('withHeap')
}
if (project.hasProperty('withThreads')) {
systemProperty 'benchmark.withThreads', project.findProperty('withThreads')
}
if (project.hasProperty('withWarmup')) {
systemProperty 'benchmark.withWarmup', project.findProperty('withWarmup')
}
if (project.hasProperty('withDuration')) {
systemProperty 'benchmark.withDuration', project.findProperty('withDuration')
}

systemProperty 'benchmark.withSsl', project.hasProperty('withSsl')
systemProperty 'benchmark.withSslProtocols', project.findProperty('withSslProtocols')
systemProperty 'benchmark.withSslCiphers', project.findProperty('withSslCiphers')

if (project.hasProperty('withSniProxy')) {
systemProperty 'benchmark.withSniProxy', project.findProperty('withSniProxy')
}
systemProperty 'benchmark.withSniProxyImage', project.findProperty('withSniProxyImage')

if (project.hasProperty('withRouter')) {
systemProperty 'benchmark.withRouter', project.findProperty('withRouter')
}
systemProperty 'benchmark.withRouterImage', project.findProperty('withRouterImage')

systemProperty 'benchmark.withSecurityManager', project.hasProperty('withSecurityManager')
legacyProperty(delegate, "withGc")
legacyProperty(delegate, "withHeap")
legacyProperty(delegate, "withThreads")
legacyProperty(delegate, "withWarmup")
legacyProperty(delegate, "withDuration")
legacyProperty(delegate, "withSsl")
legacyProperty(delegate, "withSslProtocols")
legacyProperty(delegate, "withSslCiphers")
legacyProperty(delegate, "withSniProxy")
legacyProperty(delegate, "withSniProxyImage")
legacyProperty(delegate, "withRouter")
legacyProperty(delegate, "withRouterImage")
legacyProperty(delegate, "withSecurityManager")

doFirst {
if(!project.hasProperty('hosts')) {
throw new GradleException("You must set the hosts property to a comma separated list of hosts. Eg ./gradlew benchmark -Phosts=localhost,localhost,localhost")
}
}
}

private void legacyProperty(Test test, String key) {
if (project.hasProperty(key)) {
logger.warn("-P${key} is deprecated. Use -Pbenchmark.${key}.")
test.systemProperty "benchmark.${key}", project.findProperty(key)
}
}
@@ -16,13 +16,14 @@
package org.apache.geode.benchmark.parameters;

import static java.lang.Boolean.getBoolean;
import static org.apache.geode.benchmark.parameters.JavaVersion.v11;
import static org.apache.geode.benchmark.parameters.Utils.configureGeodeProductJvms;
import static org.apache.geode.perftest.jvms.JavaVersion.v11;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.apache.geode.perftest.TestConfig;
import org.apache.geode.perftest.jvms.JavaVersion;

public class GcLoggingParameters {
private static final Logger logger = LoggerFactory.getLogger(GcLoggingParameters.class);
@@ -15,74 +15,86 @@

package org.apache.geode.benchmark.parameters;

import static org.apache.geode.benchmark.parameters.JavaVersion.v11;
import static org.apache.geode.benchmark.parameters.Utils.configureGeodeProductJvms;
import static java.lang.System.getProperty;
import static org.apache.geode.benchmark.Config.jvmArgs;
import static org.apache.geode.benchmark.topology.RoleKinds.GEODE_PRODUCT;
import static org.apache.geode.benchmark.topology.Roles.rolesFor;
import static org.apache.geode.perftest.jvms.JavaVersion.v11;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.apache.geode.benchmark.topology.Roles;
import org.apache.geode.perftest.TestConfig;
import org.apache.geode.perftest.jvms.JavaVersion;

public class GcParameters {
private static final Logger logger = LoggerFactory.getLogger(GcParameters.class);

public static void configure(final TestConfig testConfig) {
final GcImplementation gcImplementation =
GcImplementation.valueOf(System.getProperty("benchmark.withGc", "CMS"));
logger.info("Configuring {} GC.", gcImplementation);
switch (gcImplementation) {
case CMS:
configureCms(testConfig);
break;
case G1:
configureG1(testConfig);
break;
case Z:
configureZ(testConfig);
break;
case Shenandoah:
configureShenandoah(testConfig);
break;
case Epsilon:
configureEpsilon(testConfig);
break;
}
final String defaultGc = getProperty("benchmark.withGc", "CMS");

rolesFor(GEODE_PRODUCT).forEach(role -> {
final String roleName = role.toString().toLowerCase();
final GcImplementation gcImplementation =
GcImplementation.valueOf(getProperty("benchmark." + roleName + ".withGc", defaultGc));

logger.info("Configuring {} with {} GC.", roleName, gcImplementation);
switch (gcImplementation) {
case CMS:
configureCms(testConfig, role);
break;
case G1:
configureG1(testConfig, role);
break;
case Z:
configureZ(testConfig, role);
break;
case Shenandoah:
configureShenandoah(testConfig, role);
break;
case Epsilon:
configureEpsilon(testConfig, role);
break;
}

});
}

private static void configureEpsilon(final TestConfig testConfig) {
configureGeodeProductJvms(testConfig,
private static void configureEpsilon(final TestConfig testConfig, final Roles role) {
jvmArgs(testConfig, role,
"-XX:+UnlockExperimentalVMOptions",
"-XX:+UseEpsilonGC",
"-XX:+UseNUMA");
}

private static void configureShenandoah(final TestConfig testConfig) {
configureGeodeProductJvms(testConfig,
private static void configureShenandoah(final TestConfig testConfig, final Roles role) {
jvmArgs(testConfig, role,
"-XX:+UnlockExperimentalVMOptions",
"-XX:+UseShenandoahGC",
"-XX:+AlwaysPreTouch",
"-XX:+UseNUMA");
}

private static void configureZ(final TestConfig testConfig) {
private static void configureZ(final TestConfig testConfig, final Roles role) {
final JavaVersion javaVersion = JavaVersion.current();
if (javaVersion.olderThan(v11)) {
throw new IllegalArgumentException("ZGC requires Java 11 or newer");
}
configureGeodeProductJvms(testConfig,
jvmArgs(testConfig, role,
"-XX:+UnlockExperimentalVMOptions",
"-XX:+UseZGC");
}

private static void configureG1(final TestConfig testConfig) {
configureGeodeProductJvms(testConfig,
private static void configureG1(final TestConfig testConfig, final Roles role) {
jvmArgs(testConfig, role,
"-XX:+UseG1GC",
"-XX:+UseNUMA",
"-XX:+ScavengeBeforeFullGC");
}

private static void configureCms(final TestConfig testConfig) {
configureGeodeProductJvms(testConfig,
private static void configureCms(final TestConfig testConfig, final Roles role) {
jvmArgs(testConfig, role,
"-XX:+UseConcMarkSweepGC",
"-XX:+UseCMSInitiatingOccupancyOnly",
"-XX:+CMSClassUnloadingEnabled",

0 comments on commit fb25358

Please sign in to comment.