-
Notifications
You must be signed in to change notification settings - Fork 748
[GOBBLIN-1505] Add metrics for WorkUnit count #3351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3351 +/- ##
============================================
- Coverage 46.43% 46.41% -0.02%
- Complexity 10120 10155 +35
============================================
Files 2052 2056 +4
Lines 79656 80102 +446
Branches 8896 8950 +54
============================================
+ Hits 36991 37183 +192
- Misses 39226 39472 +246
- Partials 3439 3447 +8
Continue to review full report at Codecov.
|
|
@arjun4084346 @jack-moseley can you review? |
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixJobLauncher.java
Outdated
Show resolved
Hide resolved
f8aaa7b to
d9fe511
Compare
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixJobLauncher.java
Outdated
Show resolved
Hide resolved
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixJobLauncher.java
Outdated
Show resolved
Hide resolved
16e5839 to
63ede5f
Compare
*Emitted in the form <flowgroup>.<flowname>.WorkUnitsCreated
63ede5f to
363c8e7
Compare
|
+1 LGTM but not sure about the necessity of this metrics. |
|
@sv2000 can you merge? |
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/mapreduce/MRJobLauncher.java
Outdated
Show resolved
Hide resolved
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/mapreduce/MRJobLauncher.java
Outdated
Show resolved
Hide resolved
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/mapreduce/MRJobLauncher.java
Outdated
Show resolved
Hide resolved
…across ingestion modes
...n-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaAvroJobStatusMonitor.java
Outdated
Show resolved
Hide resolved
…across ingestion modes
…across ingestion modes
...n-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaAvroJobStatusMonitor.java
Outdated
Show resolved
Hide resolved
...n-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaAvroJobStatusMonitor.java
Outdated
Show resolved
Hide resolved
|
@aplex can you merge? |
...n-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaAvroJobStatusMonitor.java
Outdated
Show resolved
Hide resolved
...n-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaAvroJobStatusMonitor.java
Outdated
Show resolved
Hide resolved
...n-service/src/main/java/org/apache/gobblin/service/monitoring/KafkaAvroJobStatusMonitor.java
Outdated
Show resolved
Hide resolved
… threadsafe hashmap
From a user/platform perspective, we want to see how many workunits are being created for each flow. The metric of workunit count will be emitted in the form [flow-group].[flow-name].workunits.
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
Description
From a user/platform perspective, we want to see how many workunits are being created for each flow. The metric of workunit count will be emitted in the form ..workunits.
Tests
Tested with flows launched on gaas to see log statement "Reporting metrics for host: {}, app: {} to Ingraphs with reporting interval of {} seconds."
Commits