[HUDI-7337] Implement MetricsReporter that reports metrics to M3 #10565
Conversation
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/m3/M3MetricsReporter.java
Show resolved
Hide resolved
| @@ -120,6 +120,16 @@ | |||
| <groupId>io.prometheus</groupId> | |||
| <artifactId>simpleclient_pushgateway</artifactId> | |||
| </dependency> | |||
| <dependency> | |||
| <groupId>com.uber.m3</groupId> | |||
| <artifactId>tally-m3</artifactId> | |||
There was a problem hiding this comment.
Is there any risk for the uber specific jar to conflict with others?
There was a problem hiding this comment.
I took a look at dependencies used by com.uber.m3:tally-m3 and com.uber.m3:tally-core . It seems that the former has dependency org.apache.thrift:libthrift:jar:0.9.3 and com.google.code.findbugs:jsr305:jar:3.0.2 .
I'll take a look at how to make sure com.uber.m3:tally-m3 uses its own version of thrift and com.google.code.findbugs so that HUDI APIs (and other dependencies) won't use in tally's versions and tally will use the versions it needs, which I think I might need to use shading to solve.
There was a problem hiding this comment.
For now I ended up shading com.uber.m3 in the packaging/hudi-*-bundle POM files. To be fully transparent, I'm not too familiar with MVN/Java dependency management, so I did this approach based on what I saw on the web. Since my main priority is to just make sure dependencies "brought in" by com.uber.m3 don't "override" jar versions by hoodie_oss or users. If there's a better convention/technique for guarding against future dependency conflicts in Apache Java projects then I can try doing that instead.
prashantwason
added
the
release-0.14.2
| @@ -120,6 +120,16 @@ | |||
| <groupId>io.prometheus</groupId> | |||
| <artifactId>simpleclient_pushgateway</artifactId> | |||
| </dependency> | |||
| <dependency> | |||
| <groupId>com.uber.m3</groupId> | |||
| <artifactId>tally-m3</artifactId> | |||
Change Logs
com.uber.m3:tally-m3,com.uber.m3:tally-coreImpact
hoodie.metrics.reporter.typeasM3and their corresponding host address/port inhoodie.metrics.m3.host/hoodie.metrics.m3.portRisk level (write none, low medium or high below)
low
Documentation Update
Documentation should be updated with the new config options for M3 metrics reporting:
hoodie.metrics.m3.hosthoodie.metrics.m3.porthoodie.metrics.m3.tagshoodie.metrics.m3.servicehoodie.metrics.m3.envContributor's checklist