Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]DataProxy support monitor indicator with JMX. #1796

Closed
2 tasks done
luchunliang opened this issue Nov 15, 2021 · 0 comments · Fixed by #1797
Closed
2 tasks done

[Feature]DataProxy support monitor indicator with JMX. #1796

luchunliang opened this issue Nov 15, 2021 · 0 comments · Fixed by #1797

Comments

@luchunliang
Copy link
Contributor

luchunliang commented Nov 15, 2021

Description

[Feature]DataProxy provide monitor indicator based on JMX, user can implement the code that read the metrics and report to user-defined monitor system.

DataProxy provide monitor indicator based on JMX, user can implement the code that read the metrics and report to user-defined monitor system.
Source-module and Sink-module can add monitor metric class that is the subclass of org.apache.inlong.commons.config.metrics.MetricItemSet, and register it to MBeanServer.
User-defined plugin can get module metric with JMX, and report metric data to different monitor system.

User can describe the configuration in the file "common.properties ".
For example:
metricDomains=DataProxy metricDomains.DataProxy.domainListeners=com.tencent.pcg.atta.dataproxy.metrics.m007.M007MetricListener org.apache.inlong.dataproxy.metrics.prometheus.PrometheusMetricListener metricDomains.DataProxy.snapshotInterval=60000

The JMX domain name of DataProxy is "DataProxy".
It is defined by the parameter "metricDomains".
The listeners of JMX domain is defined by the parameter "metricDomains.$domainName.domainListeners".
The class names of the listeners is separated by the space char.
The listener class need to implement the interface "org.apache.inlong.dataproxy.metrics.MetricListener".
The snapshot interval of the listeners is defined by the parameter "metricDomains.$domainName.snapshotInterval", the parameter unit is "millisecond".

The method proto of org.apache.inlong.dataproxy.metrics.MetricListener is:
public void snapshot(String domain, List itemValues);

The field of MetricItemValue.dimensions has these key(The fields of DataProxyMetricItem defined by the Annotation "@Dimension"):

  • public String clusterId;
  • public String sourceId;
  • public String sourceDataId;
  • public String inlongGroupId;
  • public String inlongStreamId;
  • public String sinkId;
  • public String sinkDataId;

The field of MetricItemValue.metrics has these key(The fields of DataProxyMetricItem defined by the Annotation "@CountMetric"):

  • readSuccessCount
  • readSuccessSize
  • readSuccessCount
  • readSuccessSize
  • readFailCount
  • readFailSize
  • sendCount
  • sendSize
  • sendSuccessCount
  • sendSuccessSize
  • sendFailCount
  • sendFailSize
  • sinkDuration, the unit is millisecond, the duration is between current timepoint and the timepoint in sending to sink destination.
  • nodeDuration, the unit is millisecond, the duration is between current timepoint and the timepoint in getting event from source.
  • wholeDuration, the unit is millisecond, the duration is between current timepoint and the timepoint in generating event.

Use case

No response

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

aloyszhang pushed a commit that referenced this issue Nov 18, 2021
* [Feature]DataProxy support monitor indicator with JMX. #1796

* [Feature]DataProxy support monitor indicator with JMX. #1796

* [Feature]DataProxy support monitor indicator with JMX. #1796

* [Feature]DataProxy support monitor indicator with JMX. #1796

* fix PR reviews.

* fix PR reviews.

* fix checkstyle problem.

* add more unit test case.

* add apache licenses.

* fix checkstyle problem.

* fix checkstyle problem.
@gosonzhang gosonzhang added this to the 0.12.0 milestone Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants