Improves on SNI proxy benchmarking. (#136)

* Adds support for Envoy as proxy server.
* Adds support for manually configured and managed proxy.
* Adds ability to switch between proxy implementations.
* Adds support for overriding docker image:tag for proxy implementations.

README.md

@@ -55,6 +55,8 @@ Options:
     -PwithSslProtocols    : Specifies enabled SSL protocols. See Geode property `ssl-protocols`
     -PwithSslCiphers      : Specifies enabled SSL chipher suites. See Geode property `ssl-ciphers`
     -PwithSecurityManager : Flag to start Geode with the example implementation of SecurityManager
+    -PwithSniProxy        : Use SNI proxy topology.
+    -PwithSniProxyImage   : Provide an alternative Docker image coordinate for SNI proxy.
     -PwithGc              : Select which GC to use. Valid values CMS (default), G1, Z.
     -PwithHeap            : Specify how large a heap the benchmark VMs should use, default "8g". Accepts any `-Xmx` value, like "32g".
     -PwithThreads         : Specify how many threads to use when executing the benchmark. Default varies by benchmark.
@@ -150,32 +152,21 @@ public class PutTask extends BenchmarkDriverAdapter implements Serializable {
 }
 ```
 
-## SNI
+## SNI Proxy
 
-On AWS, you can run any benchmark on a topology that routes all client-server communication through an SNI proxy (HAproxy).
+You can run any benchmark on a topology that routes all client-server communication through an SNI proxy.
+
+The `withSniProxy` property accepts:
+ * `HAProxy` for HAProxy based SNI proxy (default).
+ * `Envoy` for Envoy based SNI proxy.
+ * `Manual` for providing your own SNI proxy and managing its lifecycle.
+
+The `withSniProxyImage` property can be used to provide an alternative Docker image to one of the 
+supported proxy implementations. The value should be set to a valid Docker image coordinate. 
 
-To run a test, e.g. `PartitionedGetBenchmark`, with SNI:
-
-`./run_tests.sh -t anytagname -- -PwithSniProxy '--tests=PartitionedGetBenchmark'`
-
-Since SNI is a feature of TLS, running with the SNI topology incurs TLS overheads.
-
-### TODO for SNI
-* ~~verify `StartSniProxy` runs on proxy node~~
-* ~~don't require operator to supply `-PwithSSL`/`-DwithSSL=true` when running SNI tests~~
-* ~~generate `haproxy.cfg` with client-visible SNI hostnames~~
-* ~~turn on SNI via `setPoolSocketFactory` in a new `StartClientSNI` task~~
-* ~~set `--hostname-for-clients` on locator and servers for SNI~~
-* ~~reinstate thread-per-core in `PrePopulateRegion.run()` and in `PartitionedPutBenchmark[SNI]` ya~~
-* ~~set `keyRange` back to 1e6 in `PartitionedPutBenchmark[SNI]` after client-server connections are healthy~~
-* ~~make topology orthogonal to tests so all tests can run with SNI; have a `-PwithSniProxy`/`-DwithSniProxy=true` flag~~
-* Potential performance improvement: HAproxy as configured runs one process with the max threads-per-process of 64 threads, ostensibly using 64/72 cores (89%.) We might be able to improve performance by configuring HAproxy to run in daemon mode where we can run two processes, each multithreaded, to run more than 64 threads, thereby utilizing 100% of our cores.  
-
-## TODO (General)
-* add logic to clean up existing locator.dat files before running a locator on a node
-* eliminate `harness` module dependency on Geode by moving Geode keystore/truststore setting out of `harness` module and up into `geode-benchmarks` i.e. set 'em in properties sent to `Locator.startLocatorAndDS` in `StartLocator`, `StartServer`
-* move `docker-compose.yml` distribution out of `harness` module up into `geode-benchmarks` so it gets distributed whenever it changes (without requiring rebuilding AWS AMI and cluster on AWS) 
-* generate 2048-bit keys (instead of 1024-bit ones) for TLS; will slow TLS handshakes which may necessitate a new baseline
-* make `StartServer` task use `ServerLauncher` (instead of `CacheFactory`) for symmetry with `LocatorLauncher`—also too: encapsulation!
-* `./run_tests.sh` sometimes seems to hang after benchmarks have completed, requiring operator to enter ^C to un-stick it
-* make `rsync:` Git "scheme" work in `run_tests.sh` script for benchmark repo (not just for geode repo)
+To run a test, e.g. `PartitionedGetBenchmark`, with HAProxy SNI Proxy:
+
+`./run_tests.sh -t anytagname -- -PwithSniProxy=HAProxy --tests=PartitionedGetBenchmark`
+
+Since SNI is a feature of TLS, running with the SNI topology incurs TLS overheads with implied `-PwithSsl`.
+

geode-benchmarks/build.gradle

@@ -58,6 +58,7 @@ dependencies {
   // Required for missing dependency on geode-core.
   runtime(group: 'org.eclipse.jetty', name: 'jetty-webapp', version: '9.4.12.v20180830')
 
+  testImplementation(group: 'org.junit.jupiter', name: 'junit-jupiter-params', version:project.'junit-jupiter-engine.version')
   testImplementation(group: 'org.mockito', name: 'mockito-all', version: project.'mockito-all.version')
   testImplementation(group: 'org.assertj', name: 'assertj-core', version: project.'assertj-core.version')
 }
@@ -102,13 +103,20 @@ task benchmark(type: Test) {
   if (project.hasProperty('withDuration')) {
     systemProperty 'withDuration', project.findProperty('withDuration')
   }
+
   systemProperty 'withSsl', project.hasProperty('withSsl')
   systemProperty 'withSslProtocols', project.findProperty('withSslProtocols')
   systemProperty 'withSslCiphers', project.findProperty('withSslCiphers')
-  systemProperty 'withSniProxy', project.hasProperty('withSniProxy')
+
+  if (project.hasProperty('withSniProxy')) {
+    systemProperty 'withSniProxy', project.findProperty('withSniProxy')
+  }
+  systemProperty 'withSniProxyImage', project.findProperty('withSniProxyImage')
+
   systemProperty 'withSecurityManager', project.hasProperty('withSecurityManager')
   systemProperty 'benchmark.profiler.argument', project.findProperty('benchmark.profiler.argument')
 
+
   doFirst {
     if(!project.hasProperty('hosts')) {
       throw new GradleException("You must set the hosts property to a comma separated list of hosts. Eg ./gradlew benchmark -Phosts=localhost,localhost,localhost")

geode-benchmarks/src/main/java/org/apache/geode/benchmark/tasks/StartClient.java

@@ -23,7 +23,6 @@
 import java.io.File;
 import java.lang.reflect.InvocationTargetException;
 import java.net.InetAddress;
-import java.net.UnknownHostException;
 import java.util.Properties;
 
 import org.apache.geode.cache.client.ClientCache;
@@ -67,12 +66,12 @@ protected ClientCacheFactory createClientCacheFactory(final InetAddress locator,
       final String statsFile,
       final Properties properties,
       final TestContext context)
-      throws UnknownHostException, NoSuchMethodException, InvocationTargetException,
+      throws NoSuchMethodException, InvocationTargetException,
       IllegalAccessException, ClassNotFoundException {
     return new ClientCacheFactory(properties)
         .setPdxSerializer(new ReflectionBasedAutoSerializer("benchmark.geode.data.*"))
         .setPoolIdleTimeout(-1)
         .set(ConfigurationProperties.STATISTIC_ARCHIVE_FILE, statsFile)
-        .addPoolLocator(locator.getHostAddress(), locatorPort);
+        .addPoolLocator(locator.getHostName(), locatorPort);
   }
 }

geode-benchmarks/src/main/java/org/apache/geode/benchmark/tasks/StartClientSNI.java

@@ -17,55 +17,44 @@
 
 package org.apache.geode.benchmark.tasks;
 
-import static org.apache.geode.benchmark.tasks.DefineHostNamingsOffPlatformTask.getOffPlatformHostName;
-import static org.apache.geode.benchmark.topology.Ports.SNI_PROXY_PORT;
-import static org.apache.geode.benchmark.topology.Roles.LOCATOR;
 import static org.apache.geode.benchmark.topology.Roles.PROXY;
 
 import java.lang.reflect.InvocationTargetException;
 import java.lang.reflect.Method;
 import java.net.InetAddress;
-import java.net.UnknownHostException;
 import java.util.Properties;
 
 import org.apache.geode.cache.client.ClientCacheFactory;
-import org.apache.geode.distributed.ConfigurationProperties;
-import org.apache.geode.pdx.ReflectionBasedAutoSerializer;
 import org.apache.geode.perftest.TestContext;
 
 public class StartClientSNI extends StartClient {
 
-  public StartClientSNI(final int locatorPort) {
+  private final int proxyPort;
+
+  public StartClientSNI(final int locatorPort, final int proxyPort) {
     super(locatorPort);
+    this.proxyPort = proxyPort;
   }
 
   @Override
   protected ClientCacheFactory createClientCacheFactory(final InetAddress locator,
       final String statsFile,
       final Properties properties,
       final TestContext context)
-      throws UnknownHostException, NoSuchMethodException, InvocationTargetException,
+      throws NoSuchMethodException, InvocationTargetException,
       IllegalAccessException, ClassNotFoundException {
 
-    final InetAddress firstLocatorAddy =
-        context.getHostsForRole(LOCATOR.name()).iterator().next();
-    final String offPlatformLocatorName =
-        getOffPlatformHostName(context, firstLocatorAddy);
-    final InetAddress proxyAddy =
-        context.getHostsForRole(PROXY.name()).iterator().next();
+    final ClientCacheFactory cacheFactory =
+        super.createClientCacheFactory(locator, statsFile, properties, context);
 
-    final ClientCacheFactory cacheFactory = new ClientCacheFactory(properties)
-        .setPdxSerializer(new ReflectionBasedAutoSerializer("benchmark.geode.data.*"))
-        .setPoolIdleTimeout(-1)
-        .set(ConfigurationProperties.STATISTIC_ARCHIVE_FILE, statsFile)
-        .addPoolLocator(offPlatformLocatorName, locatorPort);
-    final String proxyHostAddress = proxyAddy.getHostAddress();
-    return reflectivelySetSniSocketFactory(cacheFactory, proxyHostAddress);
+    final InetAddress proxyInetAddress =
+        context.getHostsForRole(PROXY.name()).stream().findFirst().get();
+    return reflectivelySetSniSocketFactory(cacheFactory, proxyInetAddress);
   }
 
   protected ClientCacheFactory reflectivelySetSniSocketFactory(
       final ClientCacheFactory clientCacheFactory,
-      final String proxyHostAddress)
+      final InetAddress proxyInetAddress)
       throws NoSuchMethodException, InvocationTargetException, IllegalAccessException,
       ClassNotFoundException {
     /*
@@ -83,7 +72,8 @@ protected ClientCacheFactory reflectivelySetSniSocketFactory(
     final Method sniStaticMethod =
         proxySocketFactoriesClass.getMethod("sni", String.class, int.class);
 
-    final Object sniSocketFactory = sniStaticMethod.invoke(null, proxyHostAddress, SNI_PROXY_PORT);
+    final Object sniSocketFactory =
+        sniStaticMethod.invoke(null, proxyInetAddress.getHostName(), proxyPort);
 
     final Class<?> socketFactoryClass =
         Class.forName("org.apache.geode.cache.client.SocketFactory");