Skip to content

Consolidate pauseless ingestion failure tests - #19192

Closed
xiangfu0 wants to merge 2 commits into
apache:masterfrom
xiangfu0:xiangfu0/codex/consolidate-pauseless-ingestion-tests
Closed

Consolidate pauseless ingestion failure tests#19192
xiangfu0 wants to merge 2 commits into
apache:masterfrom
xiangfu0:xiangfu0/codex/consolidate-pauseless-ingestion-tests

Conversation

@xiangfu0

@xiangfu0 xiangfu0 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • consolidate the four pauseless controller-failure integration classes into one shared-cluster class
  • publish the Kafka dataset and build the non-pauseless reference table once, while keeping one unique table/schema and one independently reported test per scenario
  • preserve the baseline, commit-end metadata, ideal-state update, and new-segment metadata assertions and recovery flow
  • make repeated fault injection thread-safe and force segment-level validation for every explicit recovery
  • keep the existing two integration-test jobs and alphabetical Set 2 selection unchanged

Why

Each former concrete class started and stopped ZooKeeper, Kafka, controller, broker, and server, then republished the same 115,545 rows and rebuilt the same reference table.

In successful workflow run 31227570484, the four classes took 476.1 seconds combined:

  • baseline: 109.8s
  • commit-end metadata failure: 123.0s
  • ideal-state update failure: 122.8s
  • new-segment metadata failure: 120.5s

The consolidated local class completed all four tests in 103.6 seconds. This is a directional comparison across different runners; this PR's Set 2 CI result will provide the apples-to-apples workflow measurement.

Correctness details

  • Fault scenarios are serialized because controller failure injection is global.
  • Every scenario disables its fault in cleanup, drops the table, waits for ExternalView and server table-manager removal, and deletes its schema.
  • Cleanup failures are suppressed onto the original test failure instead of masking it.
  • The decoder schema file is pinned for the shared fixture lifetime so later Helix transitions cannot observe a deleted class temp file.
  • Repeated recovery explicitly requests segment-level validation. A shared validator otherwise rate-limits later deep-store upload passes through its last-run timestamp.
  • The topology-changing TableRebalancePauselessIntegrationTest remains dedicated and passes unchanged.

Validation

  • ./mvnw -pl pinot-integration-tests -am -Dtest=PauselessRealtimeIngestionIntegrationTest -Dsurefire.failIfNoSpecifiedTests=false test
    • 4 tests, 0 failures/errors/skips, 103.6s class time
  • ./mvnw -pl pinot-integration-tests -am -Dtest=TableRebalancePauselessIntegrationTest -Dsurefire.failIfNoSpecifiedTests=false test
    • 2 tests, 0 failures/errors/skips, 131.4s class time
  • ./mvnw spotless:apply -pl pinot-controller,pinot-integration-tests
  • ./mvnw license:format -pl pinot-controller,pinot-integration-tests
  • ./mvnw checkstyle:check -pl pinot-controller,pinot-integration-tests
  • ./mvnw license:check -pl pinot-controller,pinot-integration-tests

@codecov-commenter

codecov-commenter commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.65%. Comparing base (baaf0a7) to head (ae04264).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19192      +/-   ##
============================================
+ Coverage     66.62%   66.65%   +0.02%     
  Complexity     1423     1423              
============================================
  Files          3443     3443              
  Lines        218626   218632       +6     
  Branches      34792    34793       +1     
============================================
+ Hits         145662   145729      +67     
+ Misses        61240    61181      -59     
+ Partials      11724    11722       -2     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-25 66.65% <ø> (+0.02%) ⬆️
temurin 66.65% <ø> (+0.02%) ⬆️
unittests 66.65% <ø> (+0.02%) ⬆️
unittests1 57.23% <ø> (+0.03%) ⬆️
unittests2 38.90% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xiangfu0

xiangfu0 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #19190.

@xiangfu0 xiangfu0 closed this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants