Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail reload if derived columns can't be created #11559

Merged
merged 1 commit into from Sep 12, 2023

Conversation

saurabhd336
Copy link
Contributor

Currently, a reload operation silently fails without errors when a newly added derived column can't be created due to transform function failures. This change makes sure we always fail a reload op with an error if newly added derived columns can't be created. If for whatever reason this behaviour is desirable for certain reload ops, we may add a new option (like forceDownload) to the reload API to toggle.

@codecov-commenter
Copy link

codecov-commenter commented Sep 11, 2023

Codecov Report

Merging #11559 (f610a8a) into master (2ab2532) will increase coverage by 0.03%.
Report is 33 commits behind head on master.
The diff coverage is 12.50%.

@@             Coverage Diff              @@
##             master   #11559      +/-   ##
============================================
+ Coverage     63.03%   63.07%   +0.03%     
- Complexity     1108     1109       +1     
============================================
  Files          2320     2325       +5     
  Lines        124528   124787     +259     
  Branches      19011    19056      +45     
============================================
+ Hits          78496    78706     +210     
- Misses        40437    40480      +43     
- Partials       5595     5601       +6     
Flag Coverage Δ
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 63.01% <12.50%> (+<0.01%) ⬆️
java-17 62.90% <12.50%> (+<0.01%) ⬆️
java-20 62.93% <12.50%> (+62.93%) ⬆️
temurin 63.07% <12.50%> (+0.03%) ⬆️
unittests 63.06% <12.50%> (+0.03%) ⬆️
unittests1 67.48% <15.38%> (-0.05%) ⬇️
unittests2 14.48% <0.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...server/starter/helix/HelixInstanceDataManager.java 0.00% <0.00%> (ø)
...loader/defaultcolumn/BaseDefaultColumnHandler.java 56.09% <10.00%> (-1.16%) ⬇️
...local/segment/index/loader/IndexLoadingConfig.java 84.05% <33.33%> (-0.38%) ⬇️

... and 45 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@xiangfu0 xiangfu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I just observed this issue when adding a column with sorted values into inverted columns and try to reload the segment.

@saurabhd336 saurabhd336 merged commit bc0d9fb into apache:master Sep 12, 2023
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants