Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SegmentZKMetadata time handling #7375

Merged

Conversation

Jackie-Jiang
Copy link
Contributor

On top of #7372, fix the getTimeInterval() and minor performance improvement. Also add some comments for future reference

@jasperjiaguo
Copy link
Contributor

Check-style won't let you do the inline assignment. I got red flag as well so had to cast twice.

@jasperjiaguo
Copy link
Contributor

Thanks for fixing getTimeInterval()

Copy link
Contributor

@snleee snleee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please double-check on the unit tests that are failing.

TimeUnit timeUnit = TimeUnit.valueOf(_simpleFields.get(Segment.TIME_UNIT));
return new Interval(timeUnit.toMillis(Long.parseLong(startTimeString)),
timeUnit.toMillis(Long.parseLong(endTimeString)));
long startTimeMs = getStartTimeMs();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it is deprecated, and my IDE did not find any callers. If so, may be just remove?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will remove all the deprecated ones in a separate PR so that they are easier to track

Copy link
Contributor

@jackjlli jackjlli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor but LGTM.

// NOTE: Need to check whether the start time is positive because some old segment ZK metadata contains negative
// start time and null time unit
long startTime;
if (startTimeString != null && (startTime = Long.parseLong(startTimeString)) > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the build isn't happy about inner assignment. Maybe adding a private method to extract the field helps?

@Jackie-Jiang Jackie-Jiang force-pushed the segment_zk_metadata_time_handling branch from 7311f69 to 23af1f7 Compare August 30, 2021 18:23
@codecov-commenter
Copy link

codecov-commenter commented Aug 30, 2021

Codecov Report

Merging #7375 (4b0791e) into master (2302bd2) will decrease coverage by 41.25%.
The diff coverage is 50.00%.

❗ Current head 4b0791e differs from pull request most recent head 23af1f7. Consider uploading reports for the commit 23af1f7 to get more accurate results
Impacted file tree graph

@@              Coverage Diff              @@
##             master    #7375       +/-   ##
=============================================
- Coverage     71.72%   30.46%   -41.26%     
+ Complexity     3309       88     -3221     
=============================================
  Files          1515     1506        -9     
  Lines         75008    74664      -344     
  Branches      10910    10874       -36     
=============================================
- Hits          53798    22749    -31049     
- Misses        17581    49909    +32328     
+ Partials       3629     2006     -1623     
Flag Coverage Δ
integration1 30.46% <50.00%> (-0.12%) ⬇️
integration2 ?
unittests1 ?
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...not/common/metadata/segment/SegmentZKMetadata.java 70.41% <50.00%> (-2.32%) ⬇️
...c/main/java/org/apache/pinot/common/tier/Tier.java 0.00% <0.00%> (-100.00%) ⬇️
.../java/org/apache/pinot/spi/utils/BooleanUtils.java 0.00% <0.00%> (-100.00%) ⬇️
...ava/org/apache/pinot/spi/data/MetricFieldSpec.java 0.00% <0.00%> (-100.00%) ⬇️
...va/org/apache/pinot/spi/utils/BigDecimalUtils.java 0.00% <0.00%> (-100.00%) ⬇️
...java/org/apache/pinot/common/tier/TierFactory.java 0.00% <0.00%> (-100.00%) ⬇️
...a/org/apache/pinot/spi/config/table/TableType.java 0.00% <0.00%> (-100.00%) ⬇️
.../org/apache/pinot/spi/data/DimensionFieldSpec.java 0.00% <0.00%> (-100.00%) ⬇️
.../org/apache/pinot/spi/data/readers/FileFormat.java 0.00% <0.00%> (-100.00%) ⬇️
...org/apache/pinot/spi/config/table/QuotaConfig.java 0.00% <0.00%> (-100.00%) ⬇️
... and 1038 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2302bd2...23af1f7. Read the comment docs.

@Jackie-Jiang Jackie-Jiang merged commit d412ecf into apache:master Aug 30, 2021
@Jackie-Jiang Jackie-Jiang deleted the segment_zk_metadata_time_handling branch August 30, 2021 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants