Skip to content

Enables auto reset of error segments by default#15736

Merged
Jackie-Jiang merged 1 commit intoapache:masterfrom
noob-se7en:enable_auto_reset
Jul 29, 2025
Merged

Enables auto reset of error segments by default#15736
Jackie-Jiang merged 1 commit intoapache:masterfrom
noob-se7en:enable_auto_reset

Conversation

@noob-se7en
Copy link
Contributor

Small change to always reset error segments in Realtime Segment Validation Manager.

I have asked the reason of keeping this feature behind the flag here #14217 (comment) which I feel should always be enabled without any config.

@noob-se7en
Copy link
Contributor Author

This was discussed here: #15652 (comment) as well

@codecov-commenter
Copy link

codecov-commenter commented May 7, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.26%. Comparing base (1a476de) to head (4d97fd9).
Report is 45 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #15736      +/-   ##
============================================
+ Coverage     62.90%   63.26%   +0.36%     
- Complexity     1386     1388       +2     
============================================
  Files          2867     2884      +17     
  Lines        163354   164953    +1599     
  Branches      24952    25202     +250     
============================================
+ Hits         102755   104359    +1604     
+ Misses        52847    52724     -123     
- Partials       7752     7870     +118     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.19% <ø> (+0.32%) ⬆️
java-21 63.23% <ø> (+0.41%) ⬆️
skip-bytebuffers-false ?
skip-bytebuffers-true ?
temurin 63.26% <ø> (+0.36%) ⬆️
unittests 63.26% <ø> (+0.36%) ⬆️
unittests1 56.33% <ø> (+0.51%) ⬆️
unittests2 33.47% <ø> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jackie-Jiang
Copy link
Contributor

This feature is relatively new (introduced after 1.3.0), and it also applies to offline table. It is safer to test it out first then make it default

@Jackie-Jiang Jackie-Jiang added documentation Configuration Config changes (addition/deletion/change in behavior) labels May 7, 2025
@9aman
Copy link
Contributor

9aman commented Jul 25, 2025

@Jackie-Jiang I think we can enable it by default.

I don't see any downside here. The case for pauseless is separately handled.

This will only be a concern for dedup and partial upsert tables in case an ONLINE segment is in ERROR state on one of the servers and we have incorrect metadata due to this on the failed server.

In case all the servers have it in ERROR, that most likely would happen for a CONSUMING segment having bad data or build issues. The ingestion will pause for this case and would require further debugging.

@Jackie-Jiang What concerns do we see with the OFFLINE tables ?

@Jackie-Jiang
Copy link
Contributor

@Jackie-Jiang What concerns do we see with the OFFLINE tables ?

No major concern. Since we have already tested out, agree we can enable it by default

@Jackie-Jiang Jackie-Jiang merged commit f9052de into apache:master Jul 29, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Configuration Config changes (addition/deletion/change in behavior) documentation ingestion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants