Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move logic to build a Labeled Metric Name to a util file. #30796

Merged
merged 5 commits into from Apr 15, 2024

Conversation

JayajP
Copy link
Contributor

@JayajP JayajP commented Mar 28, 2024

Move logic to build a Labeled Metric Name to a util file. Currently all of this logic is in BigQuerySinkMetrics; however, we can make use of these helper classes to add metrics on other sinks.

Additionally this CL includes a few optimizations for Labeled Metric Names described below.

Optimizing Building Metric Names:

Currently we build a labeled metric name by populating a new NavigableMap with all metric labels, and then iterating over the Map and appending to a StringBuilder. We can optimize the current method with the following:

  1. Directly append metric labels to a StringBuilder instead of first populating a NavigableMap (supported with a new wrapper class on StringBuilder, MetricNameBuilder).
  2. Directly append MetricLabels to the StringBuilder instead of first concatenating the <Label value, Label key> pair src.

Optimizing Parsing Metric Names:

Make use of Splittler.MapSplitter to parse labels from a mangled metric name.

Some quick load tests showed that this makes creating/parsing labeled metrics 33%/15% faster respectively.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Copy link

codecov bot commented Mar 29, 2024

Codecov Report

Attention: Patch coverage is 87.50000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 70.86%. Comparing base (e894d8c) to head (ebf509e).
Report is 19 commits behind head on master.

❗ Current head ebf509e differs from pull request most recent head ed0dc2e. Consider uploading reports for the commit ed0dc2e to get more accurate results

Files Patch % Lines
.../beam/sdk/io/gcp/bigquery/BigQuerySinkMetrics.java 86.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #30796      +/-   ##
============================================
- Coverage     71.47%   70.86%   -0.61%     
- Complexity        0     2990    +2990     
============================================
  Files           710     1062     +352     
  Lines        104815   132697   +27882     
  Branches          0     3230    +3230     
============================================
+ Hits          74915    94034   +19119     
- Misses        28268    35570    +7302     
- Partials       1632     3093    +1461     
Flag Coverage Δ
java 68.57% <87.50%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JayajP JayajP marked this pull request as ready for review March 29, 2024 18:19
@JayajP
Copy link
Contributor Author

JayajP commented Mar 29, 2024

R: @m-trieu

Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@JayajP JayajP changed the title Cache Histograms in BigQuerySinkMetrics Optimize histograms and metric names in BigQuerySinkMetrics Apr 1, 2024
@JayajP JayajP force-pushed the cache-v2 branch 2 times, most recently from ed0dc2e to e3f3012 Compare April 8, 2024 18:36
@codecov-commenter
Copy link

codecov-commenter commented Apr 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.08%. Comparing base (96dc16a) to head (360d26c).
Report is 2 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #30796      +/-   ##
============================================
+ Coverage     70.95%   71.08%   +0.12%     
+ Complexity     4470     2987    -1483     
============================================
  Files          1257     1063     -194     
  Lines        140917   132815    -8102     
  Branches       4305     3228    -1077     
============================================
- Hits          99989    94411    -5578     
+ Misses        37451    35312    -2139     
+ Partials       3477     3092     -385     
Flag Coverage Δ
java 68.55% <100.00%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JayajP JayajP changed the title Optimize histograms and metric names in BigQuerySinkMetrics Move logic to build a Labeled Metric Name to a util file. Apr 8, 2024
try {
Map<String, String> labels = splitter.split(metricNameSplit.get(1));
return Optional.of(ParsedMetricName.create(metricNameSplit.get(0), labels));
} catch (IllegalArgumentException e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we log something here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or maybe just move the return statement below into here to show that we only do the latter when an an exception is thrown

Copy link
Contributor

@m-trieu m-trieu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some minor nits, but other than that LGTM

@JayajP
Copy link
Contributor Author

JayajP commented Apr 12, 2024

Java IO precommit failing due to #30941

@Abacn Abacn merged commit 9fa45df into apache:master Apr 15, 2024
27 of 28 checks passed
JayajP added a commit to JayajP/beam that referenced this pull request Apr 15, 2024
* Extract logic to build labeled MetricNames to a util Class

* Address comments
bzablocki pushed a commit to bzablocki/beam that referenced this pull request Apr 16, 2024
* Extract logic to build labeled MetricNames to a util Class

* Address comments
JayajP added a commit to JayajP/beam that referenced this pull request Apr 16, 2024
* Extract logic to build labeled MetricNames to a util Class

* Address comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants