Skip to content

Commit

Permalink
deprecate master nomenclature (opensearch-project#317)
Browse files Browse the repository at this point in the history
* replace the usage of master to cluster_manager

Signed-off-by: Kaushal Kumar <kshkmr@amazon.com>
  • Loading branch information
kaushalmahi12 authored and Kaushal Kumar committed Jan 18, 2023
1 parent df79fff commit 734286f
Show file tree
Hide file tree
Showing 20 changed files with 195 additions and 179 deletions.
4 changes: 2 additions & 2 deletions docs/READER.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The performance analyzer application is written in Java, and this allows us to s

The writer has two different policies for lifecycle management of data in shared memory

* A new file is created in shared memory for every event. These files are then deleted by the writer once their age crosses a threshold. This strategy is used for http, request and master events.
* A new file is created in shared memory for every event. These files are then deleted by the writer once their age crosses a threshold. This strategy is used for http, request and cluster_manager events.
* Samples of statistics are written to a single file, which is overwritten every 5 seconds. This strategy is used for both operating system statistics(cpu, rss) and node level OpenSearch statistics(circuit breaker).

The reader scans the shared memory directory for updates every 2.5 seconds to make sure that no update from the writer is missed. This scheme avoids explicit synchronization between the reader and writer processes.
Expand Down Expand Up @@ -101,7 +101,7 @@ On a large OpenSearch cluster with heavy workload, we often see hundreds of thou

### MetricsEmitter

The metrics emitter queries in memory snapshots of data and then bulk loads them into metricsDB. This helps us process more than 100k updates per second on a single thread. A single emitter can emit multiple metrics. We currently have four emitters - request, http, node and master. Every emitter queries an inmemory snapshot and then populates the results into the corresponding metricsDB tables. The ability to add new metrics and dimensions through configuration instead of code would be a nice enhancement.
The metrics emitter queries in memory snapshots of data and then bulk loads them into metricsDB. This helps us process more than 100k updates per second on a single thread. A single emitter can emit multiple metrics. We currently have four emitters - request, http, node and cluster_manager. Every emitter queries an inmemory snapshot and then populates the results into the corresponding metricsDB tables. The ability to add new metrics and dimensions through configuration instead of code would be a nice enhancement.

### MetricsDB

Expand Down
2 changes: 1 addition & 1 deletion licenses/performanceanalyzer-rca-3.0.0.0-SNAPSHOT.jar.sha1
Original file line number Diff line number Diff line change
@@ -1 +1 @@
7f865a206b6116917c87cc794f2b19a34bb4f4b9
f6f83c4471cd122ccae11fa375c3c2e2715789a8
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Supported Elasticsearch version 7.10.0

### Features
* Publish shard state metrics ([#212](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/212))
* Add Master Throttling Collector Metrics ([#227](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/227))
* Add ClusterManager Throttling Collector Metrics ([#227](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/227))
* Publish fault detection metrics ([#218](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/218))
* Adding new GC info collector to the scheduled metrics collector([#225](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/225))

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This is the release of the Open Distro Performance Analyzer that will work with

### Improvements

* Better measurement granularity for Master Metrics [#16](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/16)
* Better measurement granularity for ClusterManager Metrics [#16](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/16)

### Bug fixes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ Version compatible with elasticsearch 6.7.1

This is the release of the Open Distro Performance Analyzer that will work with elasticsearch 6.7.1

* More Master metrics support: Master_Task_Queue_Time [#15](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/15)
* More ClusterManager metrics support: ClusterManager_Task_Queue_Time [#15](https://github.com/opendistro-for-elasticsearch/performance-analyzer/pull/15)
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Compatible with OpenSearch 1.1.0
* Use Collector override disable list for ShardIndexingPressureMetricCollector ([#28](https://github.com/opensearch-project/performance-analyzer/pull/28))
* Adding metric emission + UT for RCA_FRAMEWORK_CRASH ([#36](https://github.com/opensearch-project/performance-analyzer-rca/pull/36))
* Replace String split with Guava Splitter ([#42](https://github.com/opensearch-project/performance-analyzer-rca/pull/42))
* Add master not up writer metric ([#51](https://github.com/opensearch-project/performance-analyzer-rca/pull/51))
* Add cluster_manager not up writer metric ([#51](https://github.com/opensearch-project/performance-analyzer-rca/pull/51))

### Bug fixes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Compatible with OpenSearch 1.3.0
* Upgrade plugin to 1.3.0 and log4j to 2.17.1 ([#118](https://github.com/opensearch-project/performance-analyzer/pull/118))
* Don't run opensearch-cli in a child process ([#126](https://github.com/opensearch-project/performance-analyzer/pull/126))
* Modify grpc-netty-shaded to grpc-netty ([#129](https://github.com/opensearch-project/performance-analyzer-rca/pull/129))
* Fixes grpc channel leak issue and vertex buffer issue on non active master ([#130](https://github.com/opensearch-project/performance-analyzer-rca/pull/130))
* Fixes RCA crash on active master ([#132](https://github.com/opensearch-project/performance-analyzer-rca/pull/132))
* Fixes grpc channel leak issue and vertex buffer issue on non active cluster_manager ([#130](https://github.com/opensearch-project/performance-analyzer-rca/pull/130))
* Fixes RCA crash on active cluster_manager ([#132](https://github.com/opensearch-project/performance-analyzer-rca/pull/132))

### Maintenance

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@
import org.opensearch.performanceanalyzer.collectors.CacheConfigMetricsCollector;
import org.opensearch.performanceanalyzer.collectors.CircuitBreakerCollector;
import org.opensearch.performanceanalyzer.collectors.ClusterApplierServiceStatsCollector;
import org.opensearch.performanceanalyzer.collectors.ClusterManagerServiceEventMetrics;
import org.opensearch.performanceanalyzer.collectors.ClusterManagerServiceMetrics;
import org.opensearch.performanceanalyzer.collectors.ClusterManagerThrottlingMetricsCollector;
import org.opensearch.performanceanalyzer.collectors.DisksCollector;
import org.opensearch.performanceanalyzer.collectors.ElectionTermCollector;
import org.opensearch.performanceanalyzer.collectors.FaultDetectionMetricsCollector;
import org.opensearch.performanceanalyzer.collectors.GCInfoCollector;
import org.opensearch.performanceanalyzer.collectors.HeapMetricsCollector;
import org.opensearch.performanceanalyzer.collectors.MasterServiceEventMetrics;
import org.opensearch.performanceanalyzer.collectors.MasterServiceMetrics;
import org.opensearch.performanceanalyzer.collectors.MasterThrottlingMetricsCollector;
import org.opensearch.performanceanalyzer.collectors.NetworkInterfaceCollector;
import org.opensearch.performanceanalyzer.collectors.NodeDetailsCollector;
import org.opensearch.performanceanalyzer.collectors.NodeStatsAllShardsMetricsCollector;
Expand Down Expand Up @@ -207,9 +207,10 @@ public PerformanceAnalyzerPlugin(final Settings settings, final java.nio.file.Pa
new NodeStatsAllShardsMetricsCollector(performanceAnalyzerController));
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new NodeStatsFixedShardsMetricsCollector(performanceAnalyzerController));
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(new MasterServiceMetrics());
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new MasterServiceEventMetrics());
new ClusterManagerServiceMetrics());
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new ClusterManagerServiceEventMetrics());
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(new DisksCollector());
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new NetworkInterfaceCollector());
Expand All @@ -221,7 +222,7 @@ public PerformanceAnalyzerPlugin(final Settings settings, final java.nio.file.Pa
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new ShardStateCollector(performanceAnalyzerController, configOverridesWrapper));
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new MasterThrottlingMetricsCollector(
new ClusterManagerThrottlingMetricsCollector(
performanceAnalyzerController, configOverridesWrapper));
scheduledMetricCollectorsExecutor.addScheduledMetricCollector(
new ClusterApplierServiceStatsCollector(
Expand Down
Loading

0 comments on commit 734286f

Please sign in to comment.