test (aggregation-fuzzer): exercise spilling under a bounded memory pool - #23979
test (aggregation-fuzzer): exercise spilling under a bounded memory pool#23979naman-modi wants to merge 3 commits into
Conversation
|
Hey @2010YOUY01 this tries to close #23658, and while implementing the solution, I hit some design questions, which I wanted to discuss with you: I added a bounded-pool knob to the fuzzer, but through the SQL/SessionContext path I can't produce the plan shape the ordered streams need: sorted input always plans as a single-partition Single-mode aggregate, and multi-partition repartitions away the ordering. So under a memory limit everything falls back to the legacy So, currently, this PR exercises the legacy hash-aggregate spill path broadly (accumulators, pressure, sorted/unsorted), but not the ordered streams. what i can think of:
Which do you think about this? |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #23979 +/- ##
==========================================
- Coverage 80.85% 80.85% -0.01%
==========================================
Files 1096 1096
Lines 373911 373911
Branches 373911 373911
==========================================
- Hits 302329 302311 -18
- Misses 53547 53559 +12
- Partials 18035 18041 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Hey @2010YOUY01 / @alamb, any thoughts on this approach for fuzzy memory testing? |
c643f6d to
0e4d1ad
Compare
Which issue does this PR close?
Rationale for this change
The aggregation fuzzer always runs under the default unbounded memory pool, so it never exercises the aggregate spill paths (including the ordered-aggregation spilling from #23657).
What changes are included in this PR?
SessionContextGenerator. A context is either unbounded (today's behavior) or bounded by aFairSpillPoolthat forces a spill.peak_mem_usedmetric on the baseline plan, and bounded contexts use a range of fractions (2x, 0.5x, 0.4x, 0.334x) of it.ResourcesExhaustedand is skipped and logged, not failed.Are these changes tested?
Yes, no regression, and added a new
test_generated_context_spillsasserts a spill actually happens.Are there any user-facing changes?
No, it's Test-only.