fix(cudf): Preserve GPU batch boundaries in window spill tests - #106
Merged
sperlingxx merged 1 commit intoSep 1, 2026
Conversation
CudfFromVelox coalesces tiny Values batches up to 100000 rows, so the streaming CudfWindow disk-spill tests only saw one GPU input and one spill write. Keep gpu_batch_size_rows at 1 so each Values batch can spill independently. Fixes rapidsai#105 Part of rapidsai#100
This was referenced Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Valuesbatch as its own GPU input in the two streamingCudfWindowdisk-spill tests (velox.cudf.gpu_batch_size_rows=1).CudfFromVeloxcoalescing (100000 rows) was merging three tiny batches into one GPU input, sotestingStreamingSpillWrites()was 1 instead of >= 3.Why
AdapterOperatorTest.streamingFullPartitionCountSpillsActivePartitionandstreamingRangeSumSpillsActivePeerwere added in #20 whenValueswas still replaced byCudfValues. After GPU operators were reconciled with upstream,Valuesstays on CPU andCudfFromVeloxholds all six rows untilnoMoreInput().CudfWindowthen sees one batch and spills only the trailing group. Query results were already correct.Test plan
dev28798f623(CUDA SM 86): both tests failactual: 1 vs 3.velox_cudf_adapter_operator_test: 18/18 passed.Fixes #105
Part of #100
Prepared with assistance from Cursor.