Performance Testing Set Up For Extensibility #3012

joshpalis · 2022-04-20T20:27:16Z

The goal is to set up benchmarking for extensibility

Create a jenkins pipeline to build x64 and arm64 tarballs for both the OpenSearch feature/extensions branch and custom plugin
Spin up an m5.xlarge EC2 instance (Amazon Linux 2 AMI) to install the feature tarball into
To determine our baseline latency, using Mensor's nyc_taxis workload, conduct benchmarking on the feature tarball with the custom plugin installed
To determine latency overhead due to extensibility, using Mensor's nyc_taxis workload, conduct benchmarking on the feature tarball with the custom plugin running on a separate process

joshpalis · 2022-04-28T20:40:25Z

For future reference : Steps to set up EC-2 Instance for Extensibility performance test

EC-2 Instance should have a private IP in the CIDR range associated with the VPC, choose a CIDR range outside of the reserved CIDR blocks
Attach Mensor AccessEc2 IAM role to EC2 instance

SCP opensearch feature tarball to EC-2 instance
SSH into EC-2 instance and untar the file
Install Git : sudo yum install git -y
install Java :
- openJDK11 (java) : sudo amazon-linux-extras install java-openjdk11
- openJDK11 (javac) : sudo yum install java-11-openjdk-devel
- set JAVA HOME and PATH : in /home/ec2-user, vim ./bashrc, press i, then add these 2 lines
- export JAVA_HOME="/usr/lib/jvm/java-11-openjdk-11.0.13.0.8-1.amzn2.0.3.x86_64"
- PATH=$JAVA_HOME/bin:$PATH
- press esc key, then :wq!, then enter
- then type source .bashrc, to set JAVA_HOME and PATH
Clone custom plugin repository into EC-2 instance
Run plugin first in a background process, keep note of the PID : nohup ./gradlew run >/home/ec2-user/PluginOutput.log 2>&1 &
Run opensearch next in a background process, keep note of the PID : nohup ./bin/opensearch >/home/ec2-user/OpenSearchOutput.log 2>&1 &
Check plugin is listening on port : lsof -i :4532
Check opensearch is bound to port : lsof -i :9300
Test Put Index Action : curl -X PUT localhost:9200/single_index
Test Delete Index Action: curl -X DELETE localhost:9200/single_index
After confirmation that installation and transport communication has been established successfully, create a ticket to mensor office and provide the following information in order to peer the VPC associated with the EC-2 Instance
- Peer VPC ID
- Peer Owner ID
- Peer Role Arn
- Peer CIDR Block
- Peer Region
Verify that the VPC Peering connection has been established by checking the AWS Console and recording the VPC Peering Connection ID
Update VPC Routing tables through AWS Console and add 10.0.0.0/16 as the Destination with the VPC Peering Connection ID as the Target for every subnet within the VPC
Edit EC-2 security group inbound rules and add the following
- For Mensor : Type : All Traffic, Protocol All, Port Range All, Source 10.0.0.0/16
- For HTTP Port : Type : Custom TCP, Protocol All, Port Range 9200, Source 0.0.0.0/0
- For Security Group : Type : All Traffic, Protocol All, Port Range All, Source < Security Group ID >
Test EC-2 access via private IPv4 address
Using Mensor, set cluster endpoint to http://< EC-2 Private IPv4 address >:9200, instance type to m5.xlarge, region to us-west-2 and leave the rest of the configurations as default
Use nyc_taxis workload, 2 warm up iterations, 3 test iterations

joshpalis · 2022-05-03T18:59:56Z

Initial benchmarking results. Both tests used the nyc_taxis workload, 2 warm-up iterations, 3 test-iterations, and ran sequentially on a custom EC2 instance of type m5.xlarge

Test	Time	testExecutionId	testResults.operationsSummary.index.latencyMillis.p50	testResults.operationsSummary.index.latencyMillis.p90	testResults.operationsSummary.index.latencyMillis.p99	testResults.operationsSummary.index.latencyMillis.p100	testResults.operationsSummary.index.requestsPerSecond.mean	testResults.operationsSummary.index.requestsPerSecond.p0	testResults.operationsSummary.index.requestsPerSecond.p50	testResults.operationsSummary.index.requestsPerSecond.p100	testResults.operationsSummary.index.opCount	testResults.operationsSummary.index.opErrorCount	testResults.operationsSummary.index.opErrorRate	testResults.operationsSummary.query.latencyMillis.p50	testResults.operationsSummary.query.latencyMillis.p90	testResults.operationsSummary.query.latencyMillis.p99	testResults.operationsSummary.query.latencyMillis.p100	testResults.operationsSummary.query.requestsPerSecond.p0	testResults.operationsSummary.query.requestsPerSecond.p50	testResults.operationsSummary.query.requestsPerSecond.p100	testResults.operationsSummary.query.opCount	testResults.operationsSummary.query.opErrorCount	testResults.operationsSummary.query.opErrorRate	testResults.cpuStats.overall.p50	testResults.cpuStats.overall.p90	testResults.cpuStats.overall.p99	testResults.cpuStats.overall.p100	testResults.memoryStats.overall.p50	testResults.memoryStats.overall.p99	testResults.memoryStats.overall.p90	testResults.memoryStats.overall.p100	testResults.garbageCollection.overall.oldGCTimeMillis	testResults.garbageCollection.overall.youngGCTimeMillis
Baseline	May 2, 2022 @ 12:57:35.000	d4441ab5-231c-48cd-87de-b9811e7dd85f	3426.215	4376.37	5421.223	7388.616	22992.872	21684.729	22904.934	25746.258	47818	0	0	170.11	198.316	220.654	234.334	1.746	1.752	1.765	1510	0	0	95	95	97.73	98	46.571	77	60.2	93	0	1319846
POC	Apr 29, 2022 @ 13:20:37.000	34c8fefd-d492-420b-98a6-86daa0af7a00	2207.936	2867.279	4094.082	7383.271	36869.026	34744.883	36203.374	41631.594	46637	0	0	182.896	194.211	220.085	233.334	1.747	1.753	1.765	1510	0	0	97	98	98	98	47.667	83.04	61	90	0	1404400

dblock · 2022-05-03T20:37:51Z

What’s the code diff between baseline and POC? What are the changes at a high level?

nknize · 2022-05-03T20:44:30Z

What’s the code diff between baseline and POC? What are the changes at a high level?

https://github.com/opensearch-project/OpenSearch/compare/feature/extensions#diff-7bf77151deacfd9ad7544b9fdbd45f92857371c125950e6d34f485c7bad0b78aR23

I don't see anything in this changeset that would yield that kind throughput improvement.

joshpalis · 2022-05-03T22:42:52Z

To answer the diff between main and OpenSearch[feature/extensions] is just adding an additional consumer of onIndexModule extension point and handing it off to ExtensionsOrchestrator.
When we do manual tests, we do see about 5-50ms (depending on the instance type) of latency for each onIndexModule invocation.
Logs on OpenSearch (ran on a local developer machine)

[2022-05-03T15:02:26,816][INFO ][o.o.e.ExtensionsOrchestrator] [runTask-0] onIndexModule index:[single_index/CK-T633JSfqTF02oO8PxWQ]
[2022-05-03T15:02:26,818][INFO ][o.o.e.ExtensionsOrchestrator] [runTask-0] Sending request to extension
[2022-05-03T15:02:26,819][INFO ][o.o.t.TransportService   ] [runTask-0] TransportService:sendRequest action=indices:internal/extensions
[2022-05-03T15:02:26,819][INFO ][o.o.t.TransportService   ] [runTask-0] Action: indices:internal/extensions
[2022-05-03T15:02:26,866][INFO ][o.o.e.ExtensionsOrchestrator] [runTask-0] received IndicesModuleResponse{addIndexEventListenertrue addIndexOperationListenertrue addSearchOperationListenertrue}
[2022-05-03T15:02:26,867][INFO ][o.o.e.ExtensionsOrchestrator] [runTask-0] Received response from Extension

Our tests use OpenSearch main branch with the custom plugin installed as the baseline and OpenSearch feature/extension branch with the custom plugin running on a separate instance as our POC test.
The initial two benchmark tests use the same system and test configurations but as we are seeing unexpected latency drop we are analyzing why the latency is lower with the feature/extensions branch.
We believe the variance could be caused by different commits used on OpenSearch main vs feature/extensions.
We have started additional tests using tarballs generated by the following commits [after merging main to feature branch today]. I will post the results of these additional tests once they become available.

Baseline Test : OpenSearch main commit ID
Baseline Test : OpenSearch main input manifest
POC Test : OpenSearch feature/extensions commit ID
POC Test : OpenSearch feature/extensions input manifest
Note : both tests use Java 11, x64 architecture, m5.xlarge instance type

joshpalis · 2022-05-04T15:11:09Z

Benchmarking Test Set 2

Test	Time	testExecutionId	testResults.operationsSummary.index.latencyMillis.p50	testResults.operationsSummary.index.latencyMillis.p90	testResults.operationsSummary.index.latencyMillis.p99	testResults.operationsSummary.index.latencyMillis.p100	testResults.operationsSummary.index.requestsPerSecond.mean	testResults.operationsSummary.index.requestsPerSecond.p0	testResults.operationsSummary.index.requestsPerSecond.p50	testResults.operationsSummary.index.requestsPerSecond.p100	testResults.operationsSummary.index.opCount	testResults.operationsSummary.index.opErrorCount	testResults.operationsSummary.index.opErrorRate	testResults.operationsSummary.query.latencyMillis.p50	testResults.operationsSummary.query.latencyMillis.p90	testResults.operationsSummary.query.latencyMillis.p99	testResults.operationsSummary.query.latencyMillis.p100	testResults.operationsSummary.query.requestsPerSecond.p0	testResults.operationsSummary.query.requestsPerSecond.p50	testResults.operationsSummary.query.requestsPerSecond.p100	testResults.operationsSummary.query.opCount	testResults.operationsSummary.query.opErrorCount	testResults.operationsSummary.query.opErrorRate	testResults.cpuStats.overall.p50	testResults.cpuStats.overall.p90	testResults.cpuStats.overall.p99	testResults.cpuStats.overall.p100	testResults.memoryStats.overall.p50	testResults.memoryStats.overall.p99	testResults.memoryStats.overall.p90	testResults.memoryStats.overall.p100	testResults.garbageCollection.overall.oldGCTimeMillis	testResults.garbageCollection.overall.youngGCTimeMillis
Baseline	May 3, 2022 @ 12:42:39.000	af9ce264-30df-4661-902c-6100cb24c059	2821.6	3681.161	4527.185	6750.397	28077.893	26934.703	27749.978	30324.781	47439	0	0	142.972	148.277	167.176	184.757	1.747	1.753	1.765	1510	0	0	96	96	98	99	47	78	61	92	0	1020482
POC	May 3, 2022 @ 14:37:13.000	fe1c5df5-8ac7-4edc-8978-4a7e2504bb7a	2132.328	2803.274	3922.424	7332.989	38005.898	35592.996	37169.332	42594.754	46568	0	0	175.03	206.309	230.522	240.736	1.746	1.752	1.765	1510	0	0	97	98	98	98	47.2	78.14	62	91	0	`1332793`

joshpalis · 2022-05-04T15:13:13Z

Benchmarking Test Set 3

Test	Time	testExecutionId	testResults.operationsSummary.index.latencyMillis.p50	testResults.operationsSummary.index.latencyMillis.p90	testResults.operationsSummary.index.latencyMillis.p99	testResults.operationsSummary.index.latencyMillis.p100	testResults.operationsSummary.index.requestsPerSecond.mean	testResults.operationsSummary.index.requestsPerSecond.p0	testResults.operationsSummary.index.requestsPerSecond.p50	testResults.operationsSummary.index.requestsPerSecond.p100	testResults.operationsSummary.index.opCount	testResults.operationsSummary.index.opErrorCount	testResults.operationsSummary.index.opErrorRate	testResults.operationsSummary.query.latencyMillis.p50	testResults.operationsSummary.query.latencyMillis.p90	testResults.operationsSummary.query.latencyMillis.p99	testResults.operationsSummary.query.latencyMillis.p100	testResults.operationsSummary.query.requestsPerSecond.p0	testResults.operationsSummary.query.requestsPerSecond.p50	testResults.operationsSummary.query.requestsPerSecond.p100	testResults.operationsSummary.query.opCount	testResults.operationsSummary.query.opErrorCount	testResults.operationsSummary.query.opErrorRate	testResults.cpuStats.overall.p50	testResults.cpuStats.overall.p90	testResults.cpuStats.overall.p99	testResults.cpuStats.overall.p100	testResults.memoryStats.overall.p50	testResults.memoryStats.overall.p99	testResults.memoryStats.overall.p90	testResults.memoryStats.overall.p100	testResults.garbageCollection.overall.oldGCTimeMillis	testResults.garbageCollection.overall.youngGCTimeMillis
Baseline	May 3, 2022 @ 16:54:58.000	5d21c037-4ee9-46f2-8354-96569a18d1c5	3048.535	4013.885	4947.028	8090.254	25971.91	24854.072	25748.579	28363.842	47577	0	0	137.42	143.385	156.434	177.032	1.747	1.753	1.766	1510	0	0	95	96	97	98	46.2	76.77	61	88	0	1034909
POC	May 3, 2022 @ 17:35:07.000	d6391da1-e426-4546-bd04-00dde87f23f1	2129.366	2763.014	3968.147	7276.796	37863.043	35816.195	37216.719	42292.105	46587	0	0	177.008	208.086	223.78	241.695	1.747	1.752	1.765	1510	0	0	97	98	98	98	48	78.48	62	94	0	1351382

nknize · 2022-05-04T15:43:02Z

These numbers feel bogus. There weren't any performance improvement PRs in baseline main between 4/29 and 5/3. This looks like something is being mis-measured in the benchmarks. The extensibility branch doesn't change the indexing logic, it just connects a new node to the cluster who's sole purpose is to register an IndexModule listener. Are we sure mensor is measuring the core indexing operations and not just measuring some log events in the event listeners?

dblock · 2022-05-04T16:07:21Z

Check whether in both measurements the JDK used is the same? (on OpenSearch start header it will say what the version of Java is)

joshpalis · 2022-05-04T16:25:02Z

JDK 11 is being used for all tests. All test are run on separate EC2 instances with the same test and system configurations

saratvemulapalli · 2022-05-04T16:36:11Z

These numbers feel bogus. There weren't any performance improvement PRs in baseline main between 4/29 and 5/3. This looks like something is being mis-measured in the benchmarks. The extensibility branch doesn't change the indexing logic, it just connects a new node to the cluster who's sole purpose is to register an IndexModule listener. Are we sure mensor is measuring the core indexing operations and not just measuring some log events in the event listeners?

Totally agreed. We are surprised (rather shocked) to see improvements.
Ideally making a network call and holding the IndexModule extension request should end up in higher latency.
We'll spend sometime today to figure out the variance in our test setup. All the commits on main are merged to feature/extensions.

travisbenedict · 2022-05-04T16:36:23Z

Initial benchmarking results. Both tests used the nyc_taxis workload, 2 warm-up iterations, 3 test-iterations, and ran sequentially on a custom EC2 instance of type m5.xlarge

What is meant by "sequentially" here? Are the tests being run against the same EC2 instance with changes made between tests?

If this is the case I would try running performance tests against the POC changes running on a fresh instance. This more accurately matches the automated testing that we do where there is a new cluster created for each test

joshpalis · 2022-05-04T16:42:43Z

The first test was run on the same instance, the second and third test sets ran on their own new instances for each baseline and POC test.

dblock · 2022-05-04T17:41:41Z

JDK 11 is being used for all tests. All test are run on separate EC2 instances with the same test and system configurations

Note that we now bundle JDK17 for 2.0 by default, so I would confirm that it's actually using your global JDK11 set via JAVA_HOME.

gkamat · 2022-05-05T00:23:59Z

If the clusters are still around, we should check the opensearch logs for the latency being reported for the indexing operations on both clusters and compare those.

joshpalis · 2022-05-05T16:43:41Z

Benchmarking Test Set 4. All tests are using OpenSearch feature/extensions branch. First test uses just the feature tarball, second test uses the same feature tarball with the custom plugin installed, and the final test uses the same feature tarball with the custom plugin running on a separate process. Previous test sets used OpenSearch main branch for the baseline, however, system and test configuration for these new tests are exactly the same as the previous 3 test sets. By using the feature/extensions branch as the baseline for this set, we hope to eliminate any additional uncertainties in the results.

Test	Time	testExecutionId	testResults.operationsSummary.index.latencyMillis.p50	testResults.operationsSummary.index.latencyMillis.p90	testResults.operationsSummary.index.latencyMillis.p99	testResults.operationsSummary.index.latencyMillis.p100	testResults.operationsSummary.index.requestsPerSecond.mean	testResults.operationsSummary.index.requestsPerSecond.p0	testResults.operationsSummary.index.requestsPerSecond.p50	testResults.operationsSummary.index.requestsPerSecond.p100	testResults.operationsSummary.index.opCount	testResults.operationsSummary.query.latencyMillis.p50	testResults.operationsSummary.query.latencyMillis.p90	testResults.operationsSummary.query.latencyMillis.p99	testResults.operationsSummary.query.latencyMillis.p100	testResults.operationsSummary.query.requestsPerSecond.p0	testResults.operationsSummary.query.requestsPerSecond.p50	testResults.operationsSummary.query.requestsPerSecond.p100	testResults.operationsSummary.query.opCount	testResults.cpuStats.overall.p50	testResults.cpuStats.overall.p90	testResults.cpuStats.overall.p99	testResults.cpuStats.overall.p100	testResults.memoryStats.overall.p50	testResults.memoryStats.overall.p99	testResults.memoryStats.overall.p90	testResults.memoryStats.overall.p100	testResults.garbageCollection.overall.youngGCTimeMillis
OpenSearch [feature/extensions]	May 4, 2022 @ 17:08:12.000	01b8e71f-0760-4353-8205-31abbaabb7b0	1937.025	2514.343	3447.931	6550.258	41380.595	39431.422	40719.011	45703.508	46348	171.635	191.002	217.782	230.449	1.747	1.752	1.765	1510	97	98	98	98	48	75.17	61	90	1311769
Baseline - OpenSearch [feature/extensions] + Custom Plugin Installed	May 4, 2022 @ 14:41:57.000	621526c8-9da5-472c-aa07-c3f01b503589	3431.432	4383.891	5350.606	7723.383	23159.744	22063.988	22969.218	26059.859	47782	146.199	155.024	176.174	197.294	1.745	1.752	1.765	1510	95	95	97	98	46	66.94	60	97	1343875
POC - OpenSearch [feature/extensions], Custom Plugin on a separate process	May 4, 2022 @ 15:13:00.000	d5f71bd7-2429-4b43-a52b-e1273250ce41	2113.046	2702.456	3870.552	7148.383	38591.024	36466.57	38054.038	42708.504	46552	173.916	204.814	228.165	234.191	1.746	1.752	1.765	1510	97	98	98	98	47	75.1	61	92	`1363579`

CEHENKLE · 2022-05-05T20:41:09Z

@joshpalis @saratvemulapalli Can we run the tests on rally/OS benchmark to rule out anything wonky being added by the testing framework?

joshpalis · 2022-05-05T21:02:52Z

Here are the output logs for the POC test (d5f71bd7-2429-4b43-a52b-e1273250ce41) and the baseline test (621526c8-9da5-472c-aa07-c3f01b503589). The first 2000 lines of the opensearch output logs are pasted here, the original file is too large to post however they are still available on the EC2 instance if needed.

dblock · 2022-05-06T21:47:57Z

These numbers make more sense. The fact that "Baseline - OpenSearch [feature/extensions] + Custom Plugin Installed" is a lot slower than "POC - OpenSearch [feature/extensions], Custom Plugin on a separate process" tells me that the custom plugin introduces some kind of contention. This could be as simple as a log statement that spews a very large amount of log output that needs to be written to disk. Eliminate anything other than the boilerplate in that example, there should be almost zero overhead from the "OpenSearch [feature/extensions]" baseline.

saratvemulapalli · 2022-05-09T17:27:42Z

Thanks @dblock for taking look.
Yup we do think the logging is causing the contention which is probably causing the variance.
Assuming thats the theory we believe the tests ran so far did report accurate data but we didnt expect logging to cause a huge impact.

@joshpalis helped us kick off another set of tests, removing the INFO logging:

OpenSearch feature/extensions
OpenSearch feature/extensions with plugin
OpenSearch feature/extensions with extension.

joshpalis · 2022-05-09T17:28:58Z

Benchmarking Test Set 5. Previously, the custom plugin installed into OpenSearch logged every index event operation to an output file. Modifications to the custom plugin has been made to remove any additional logging in order to reduce the amount of information written to the disk and eliminate any performance overhead this may cause. The following three tests have been conducted using the same system and test configurations as before, with preliminary analysis of the results confirming that the latency overhead observed previously was due to the excess logging. Latency between our baseline and just opensearch are now comparable, while the POC test shows an increase in latency, which is expected.

Test	Time	testExecutionId	testResults.operationsSummary.index.latencyMillis.p50	testResults.operationsSummary.index.latencyMillis.p90	testResults.operationsSummary.index.latencyMillis.p99	testResults.operationsSummary.index.latencyMillis.p100	testResults.operationsSummary.index.requestsPerSecond.mean	testResults.operationsSummary.index.requestsPerSecond.p0	testResults.operationsSummary.index.requestsPerSecond.p50	testResults.operationsSummary.index.requestsPerSecond.p100	testResults.operationsSummary.index.opCount	testResults.operationsSummary.query.latencyMillis.p50	testResults.operationsSummary.query.latencyMillis.p90	testResults.operationsSummary.query.latencyMillis.p99	testResults.operationsSummary.query.latencyMillis.p100	testResults.operationsSummary.query.requestsPerSecond.p0	testResults.operationsSummary.query.requestsPerSecond.p50	testResults.operationsSummary.query.requestsPerSecond.p100	testResults.operationsSummary.query.opCount	testResults.cpuStats.overall.p50	testResults.cpuStats.overall.p90	testResults.cpuStats.overall.p99	testResults.cpuStats.overall.p100	testResults.memoryStats.overall.p50	testResults.memoryStats.overall.p99	testResults.memoryStats.overall.p90	testResults.memoryStats.overall.p100	testResults.garbageCollection.overall.youngGCTimeMillis
OpenSearch [feature/extensions]	May 6, 2022 @ 15:09:13.000	a30cee35-92b8-4f69-a558-3af073be8269	1916.932	2512.498	3463.146	6302.554	41786.31	39524.086	40998.286	46390.203	46292	146.715	175.709	187.188	199.805	1.747	1.753	1.765	1510	97	98	98	98	47	78.72	61	92	1348395
Baseline - OpenSearch [feature/extensions] + Custom Plugin Installed	May 6, 2022 @ 15:01:58.000	b9534aff-e7a7-4c84-add4-c1747e52f57a	1944.839	2514.855	3461.815	6927.869	41596.237	39189.352	41117.659	46011.293	46328	156.377	177.531	195.476	207.246	1.747	1.752	1.766	1510	97	98	98	98	46	82.06	61	90	1295049
POC - OpenSearch [feature/extensions], Custom Plugin on a separate process	May 6, 2022 @ 15:26:13.000	015cb631-09d6-4ff4-b8da-7817ab6a851c	2111.241	2721.908	3916.873	6792.521	38636.858	36302.305	37940.132	43614.633	46497	175.874	194.758	216.815	222.919	1.746	1.753	1.765	1510	97	98	98	98	47	66	61	91	1343277

dblock · 2022-05-10T16:07:03Z

So an 8-10% variance, that looks like a reasonable trade-off. I think we'll have to continue offering users an in-proc option.

joshpalis · 2022-05-10T17:35:01Z

Benchmarking Test Set 6. System and test configuration is the same as the previous tests, using the same feature/extensions tarball from test set 5

Test	Time	testExecutionId	testResults.operationsSummary.index.latencyMillis.p50	testResults.operationsSummary.index.latencyMillis.p90	testResults.operationsSummary.index.latencyMillis.p99	testResults.operationsSummary.index.latencyMillis.p100	testResults.operationsSummary.index.requestsPerSecond.mean	testResults.operationsSummary.index.requestsPerSecond.p0	testResults.operationsSummary.index.requestsPerSecond.p50	testResults.operationsSummary.index.requestsPerSecond.p100	testResults.operationsSummary.index.opCount	testResults.operationsSummary.query.latencyMillis.p50	testResults.operationsSummary.query.latencyMillis.p90	testResults.operationsSummary.query.latencyMillis.p99	testResults.operationsSummary.query.latencyMillis.p100	testResults.operationsSummary.query.requestsPerSecond.p0	testResults.operationsSummary.query.requestsPerSecond.p50	testResults.operationsSummary.query.requestsPerSecond.p100	testResults.operationsSummary.query.opCount	testResults.cpuStats.overall.p50	testResults.cpuStats.overall.p90	testResults.cpuStats.overall.p99	testResults.cpuStats.overall.p100	testResults.memoryStats.overall.p50	testResults.memoryStats.overall.p99	testResults.memoryStats.overall.p90	testResults.memoryStats.overall.p100	testResults.garbageCollection.overall.youngGCTimeMillis
OpenSearch [feature/extensions]	May 9, 2022 @ 15:02:12.000	2d5b053c-80c8-4803-8a98-799646ecd245	1934.522	2556.499	3553.48	6128.768	41345.591	39211.578	40599.248	45576.066	46347	142.472	148.476	161.524	176.506	1.745	1.753	1.765	1510	98	98	98	98	48	82.42	63	95	1048363
Baseline - OpenSearch [feature/extensions] + Custom Plugin Installed	May 9, 2022 @ 14:53:39.000	6b0b1da2-fc05-4c4a-81ef-23fd1a33c89b	2096.278	2772.097	3938.404	8267.693	38241	35859.949	37727.744	43139.797	46554	155.586	170.47	180.519	190.955	1.747	1.753	1.765	1510	97	98	98	99	47	69.25	61	78	1331964
POC - OpenSearch [feature/extensions], Custom Plugin on a separate process	May 9, 2022 @ 15:13:58.000	fc24bb26-66b3-46d5-8011-20b9b30cc8ba	2165.804	2756.564	3901.923	7245.699	37723.46	35593.313	37112.536	42704.164	46563	152.687	192.885	204.024	209.013	1.747	1.752	1.765	1510	97	98	98	98	48	88.18	62	96	1371192

saratvemulapalli · 2022-05-10T18:11:51Z

@joshpalis could you help understand the diff between Test Set 5 vs Test Set 6.
Also a summary would be great to have.

joshpalis added enhancement Enhancement or improvement to existing feature or request untriaged labels Apr 20, 2022

joshpalis self-assigned this Apr 20, 2022

joshpalis mentioned this issue Apr 20, 2022

Extension Latency #2231

Closed

2 tasks

saratvemulapalli added Plugins Build Libraries & Interfaces and removed untriaged labels Apr 20, 2022

owaiskazi19 mentioned this issue May 3, 2022

[feature/extensions] Updated imports with absolute path #3173

Merged

5 tasks

joshpalis closed this as completed May 10, 2022

saratvemulapalli mentioned this issue May 10, 2022

[Meta] Extensions Framework #1632

Open

16 tasks

saratvemulapalli mentioned this issue May 11, 2022

[Meta] Plugin sandboxing: Step towards modular architecture in OpenSearch #1422

Open

15 tasks

joshpalis mentioned this issue Aug 11, 2022

[META] Extensions: Performance Testing of Anomaly Detection as an Extension opensearch-project/opensearch-sdk-java#88

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance Testing Set Up For Extensibility #3012

Performance Testing Set Up For Extensibility #3012

joshpalis commented Apr 20, 2022 •

edited

Loading

joshpalis commented Apr 28, 2022 •

edited

Loading

joshpalis commented May 3, 2022 •

edited

Loading

dblock commented May 3, 2022

nknize commented May 3, 2022

joshpalis commented May 3, 2022 •

edited

Loading

joshpalis commented May 4, 2022

joshpalis commented May 4, 2022

nknize commented May 4, 2022

dblock commented May 4, 2022

joshpalis commented May 4, 2022

saratvemulapalli commented May 4, 2022

travisbenedict commented May 4, 2022

joshpalis commented May 4, 2022

dblock commented May 4, 2022

gkamat commented May 5, 2022

joshpalis commented May 5, 2022 •

edited

Loading

CEHENKLE commented May 5, 2022

joshpalis commented May 5, 2022 •

edited

Loading

dblock commented May 6, 2022 •

edited

Loading

saratvemulapalli commented May 9, 2022

joshpalis commented May 9, 2022

dblock commented May 10, 2022

joshpalis commented May 10, 2022 •

edited

Loading

saratvemulapalli commented May 10, 2022

Performance Testing Set Up For Extensibility #3012

Performance Testing Set Up For Extensibility #3012

Comments

joshpalis commented Apr 20, 2022 • edited Loading

joshpalis commented Apr 28, 2022 • edited Loading

joshpalis commented May 3, 2022 • edited Loading

dblock commented May 3, 2022

nknize commented May 3, 2022

joshpalis commented May 3, 2022 • edited Loading

joshpalis commented May 4, 2022

joshpalis commented May 4, 2022

nknize commented May 4, 2022

dblock commented May 4, 2022

joshpalis commented May 4, 2022

saratvemulapalli commented May 4, 2022

travisbenedict commented May 4, 2022

joshpalis commented May 4, 2022

dblock commented May 4, 2022

gkamat commented May 5, 2022

joshpalis commented May 5, 2022 • edited Loading

CEHENKLE commented May 5, 2022

joshpalis commented May 5, 2022 • edited Loading

dblock commented May 6, 2022 • edited Loading

saratvemulapalli commented May 9, 2022

joshpalis commented May 9, 2022

dblock commented May 10, 2022

joshpalis commented May 10, 2022 • edited Loading

saratvemulapalli commented May 10, 2022

joshpalis commented Apr 20, 2022 •

edited

Loading

joshpalis commented Apr 28, 2022 •

edited

Loading

joshpalis commented May 3, 2022 •

edited

Loading

joshpalis commented May 3, 2022 •

edited

Loading

joshpalis commented May 5, 2022 •

edited

Loading

joshpalis commented May 5, 2022 •

edited

Loading

dblock commented May 6, 2022 •

edited

Loading

joshpalis commented May 10, 2022 •

edited

Loading