Skip to content

test(spill): add unit tests and integration tests for spill-to-disk#272

Merged
lxy-9602 merged 3 commits into
alibaba:mainfrom
dalingmeng:feat/spill-test
May 15, 2026
Merged

test(spill): add unit tests and integration tests for spill-to-disk#272
lxy-9602 merged 3 commits into
alibaba:mainfrom
dalingmeng:feat/spill-test

Conversation

@dalingmeng
Copy link
Copy Markdown
Contributor

@dalingmeng dalingmeng commented May 11, 2026

Purpose

Linked issue: #149
add unit tests and integration tests for spill-to-disk

Tests

WriteInteTest.TestPkSpillable.*
KeyValueFileStoreWriteTest.TestSpill.*
MergeTreeWriterTest.TestSpill.*
WriteBufferTest.TestSpill.*
SpillReaderWriterTest

API and Format

Documentation

Generative AI tooling

Claude opus 4.6

Comment thread src/paimon/core/mergetree/merge_tree_writer_test.cpp
Comment thread src/paimon/core/mergetree/merge_tree_writer_test.cpp
Comment thread src/paimon/core/mergetree/merge_tree_writer_test.cpp
Comment thread src/paimon/core/mergetree/write_buffer_test.cpp
Comment thread src/paimon/core/mergetree/write_buffer_test.cpp
Comment thread test/inte/write_inte_test.cpp Outdated
Comment thread test/inte/write_inte_test.cpp
@lxy-9602 lxy-9602 changed the title test(spill): add unit tests and integration tests for spill-to-disk) test(spill): add unit tests and integration tests for spill-to-disk May 12, 2026
@dalingmeng dalingmeng requested a review from lxy-9602 May 12, 2026 10:39
@zjw1111 zjw1111 requested a review from Copilot May 12, 2026 10:40
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds comprehensive unit/integration coverage for spill-to-disk behavior (incl. quota exhaustion, cleanup, dedup, intermediate merge, and IO error handling), linked to issue #149.

Changes:

  • Adds new spill-focused integration tests to validate correctness, cleanup, and failure fallbacks.
  • Adds unit tests around spill file quotas, reader ordering, merge behavior, and resource cleanup.
  • Refactors repeated commit/scan logic in integration tests into helper methods.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test/inte/write_inte_test.cpp Adds PK spill integration tests + introduces Commit/Scan helper utilities to reduce duplication.
src/paimon/core/operation/key_value_file_store_write_test.cpp Adds spill tests for disk-quota fallback and multi-round dedup semantics.
src/paimon/core/mergetree/write_buffer_test.cpp Adds extensive WriteBuffer spill/quota/reader/cleanup test coverage.
src/paimon/core/mergetree/spill_reader_writer_test.cpp Adds tests for writer Close idempotency and schema mismatch error cases.
src/paimon/core/mergetree/merge_tree_writer_test.cpp Adds MergeTreeWriter spill tests: dedup, intermediate merge bounds, quota fallback, cleanup.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/inte/write_inte_test.cpp
Comment thread test/inte/write_inte_test.cpp
Comment thread test/inte/write_inte_test.cpp
Comment thread test/inte/write_inte_test.cpp
Comment thread src/paimon/core/operation/key_value_file_store_write_test.cpp
Comment thread src/paimon/core/mergetree/spill_reader_writer_test.cpp
@dalingmeng dalingmeng force-pushed the feat/spill-test branch 2 times, most recently from a035943 to 3a18972 Compare May 13, 2026 09:23
@lucasfang
Copy link
Copy Markdown
Collaborator

+1

Copy link
Copy Markdown
Collaborator

@lxy-9602 lxy-9602 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602 lxy-9602 merged commit 5af2437 into alibaba:main May 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants