Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upsert] Skip removing upsert metadata when shutting down the server #9551

Merged
merged 1 commit into from Oct 7, 2022

Conversation

Jackie-Jiang
Copy link
Contributor

When the server shuts down, removing all the metadata from the upsert manager can be time consuming and not provide any value since we re-construct the metadata during restart. This PR tracks whether the upsert manager is closed, and skips the segment remove if it is already closed

@codecov-commenter
Copy link

codecov-commenter commented Oct 7, 2022

Codecov Report

Merging #9551 (06e17c9) into master (1c202c7) will increase coverage by 7.04%.
The diff coverage is 20.00%.

@@             Coverage Diff              @@
##             master    #9551      +/-   ##
============================================
+ Coverage     62.86%   69.90%   +7.04%     
+ Complexity     5073     4876     -197     
============================================
  Files          1917     1929      +12     
  Lines        102523   102881     +358     
  Branches      15561    15600      +39     
============================================
+ Hits          64449    71921    +7472     
+ Misses        33257    25881    -7376     
- Partials       4817     5079     +262     
Flag Coverage Δ
integration1 26.05% <0.00%> (+0.03%) ⬆️
integration2 24.44% <0.00%> (-0.33%) ⬇️
unittests1 67.31% <20.00%> (+<0.01%) ⬆️
unittests2 15.70% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...cal/upsert/BasePartitionUpsertMetadataManager.java 56.77% <20.00%> (-1.63%) ⬇️
...data/manager/realtime/DefaultSegmentCommitter.java 0.00% <0.00%> (-80.00%) ⬇️
...er/api/resources/LLCSegmentCompletionHandlers.java 43.56% <0.00%> (-18.82%) ⬇️
...data/manager/realtime/SegmentCommitterFactory.java 88.23% <0.00%> (-11.77%) ⬇️
...inot/core/util/SegmentCompletionProtocolUtils.java 57.69% <0.00%> (-7.70%) ⬇️
...transform/function/IsNotNullTransformFunction.java 65.51% <0.00%> (-6.90%) ⬇️
...or/transform/function/IsNullTransformFunction.java 75.86% <0.00%> (-6.90%) ⬇️
...altime/ServerSegmentCompletionProtocolHandler.java 51.88% <0.00%> (-6.61%) ⬇️
...roller/helix/core/relocation/SegmentRelocator.java 72.97% <0.00%> (-5.41%) ⬇️
...l/segment/index/readers/OnHeapFloatDictionary.java 86.36% <0.00%> (-4.55%) ⬇️
... and 269 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@KKcorps KKcorps merged commit 8cdae92 into apache:master Oct 7, 2022
@Jackie-Jiang Jackie-Jiang deleted the accelerate_upsert_restart branch October 7, 2022 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants