Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up deprecated ZK metadata keys and methods #7846

Merged

Conversation

Jackie-Jiang
Copy link
Contributor

@Jackie-Jiang Jackie-Jiang commented Dec 1, 2021

Clean up the deprecated keys and methods as planned after releasing 0.9.0.
Also remove the deprecated and unused BackfillSegmentUtils which accesses the deprecated keys.

Upgrade Notes

To keep backward-compatibility, the cluster need to run release 0.9.0 before upgrading so that both old and new version of ZK metadata can be read.

Release Note

The following fields are removed from the segment ZK metadata

  • segment.name
  • segment.table.name
  • segment.type

The following fields are replaced with the new fields

  • segment.offline.download.url, segment.realtime.download.url -> segment.download.url
  • segment.offline.push.time -> segment.push.time
  • segment.offline.refresh.time -> segment.refresh.time

@codecov-commenter
Copy link

Codecov Report

Merging #7846 (1e3e010) into master (fc3de45) will decrease coverage by 0.06%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #7846      +/-   ##
============================================
- Coverage     71.69%   71.62%   -0.07%     
  Complexity     4089     4089              
============================================
  Files          1579     1579              
  Lines         80819    80781      -38     
  Branches      12017    12013       -4     
============================================
- Hits          57940    57862      -78     
- Misses        18971    19023      +52     
+ Partials       3908     3896      -12     
Flag Coverage Δ
integration1 29.38% <83.33%> (-0.13%) ⬇️
integration2 27.84% <83.33%> (-0.03%) ⬇️
unittests1 68.69% <100.00%> (+0.02%) ⬆️
unittests2 14.61% <50.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../core/realtime/PinotLLCRealtimeSegmentManager.java 78.97% <ø> (-0.08%) ⬇️
...lix/core/realtime/PinotRealtimeSegmentManager.java 83.76% <ø> (-1.21%) ⬇️
...ot/controller/helix/core/util/ZKMetadataUtils.java 100.00% <ø> (ø)
...manager/realtime/HLRealtimeSegmentDataManager.java 82.79% <ø> (-0.16%) ⬇️
...va/org/apache/pinot/spi/utils/CommonConstants.java 23.63% <ø> (-2.68%) ⬇️
...not/common/metadata/segment/SegmentZKMetadata.java 82.63% <100.00%> (+8.08%) ⬆️
...ntroller/helix/core/PinotHelixResourceManager.java 64.27% <100.00%> (-0.08%) ⬇️
...data/manager/realtime/DefaultSegmentCommitter.java 0.00% <0.00%> (-80.00%) ⬇️
...er/api/resources/LLCSegmentCompletionHandlers.java 43.56% <0.00%> (-18.82%) ⬇️
...data/manager/realtime/SegmentCommitterFactory.java 88.23% <0.00%> (-11.77%) ⬇️
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fc3de45...1e3e010. Read the comment docs.

@Jackie-Jiang Jackie-Jiang merged commit 098d0e7 into apache:master Dec 1, 2021
@Jackie-Jiang Jackie-Jiang deleted the segment_zk_metadata_cleanup branch December 1, 2021 18:52
@Jackie-Jiang Jackie-Jiang added the release-notes Referenced by PRs that need attention when compiling the next release notes label Dec 2, 2021
kriti-sc pushed a commit to kriti-sc/incubator-pinot that referenced this pull request Dec 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes Referenced by PRs that need attention when compiling the next release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants