Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic integration tests for WAL delta transfer #3679

Merged
merged 9 commits into from
Feb 26, 2024

Conversation

timvisee
Copy link
Member

Tracked in: #3477

This adds three basic integration tests for WAL delta transfer, similar to the tests we had for shard snapshot transfer.

The tests include:

  • transfer empty WAL delta between two nodes
  • transfer WAL delta to recover a failed node, with concurrent upsertions
  • transfer WAL delta to empty node, falling back to default transfer method

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
  3. Have you checked your code using cargo clippy --all --all-features command?

@timvisee timvisee merged commit 420d46f into dev Feb 26, 2024
17 checks passed
@timvisee timvisee deleted the shard-wal-delta-transfer-integration-tests branch February 26, 2024 10:06
timvisee added a commit that referenced this pull request Mar 5, 2024
* Add WAL delta transfer integration test for empty delta

* Report size of WAL delta, assert this in integration tests

* Add WAL delta transfer integration test for shard recovery

* Add WAL delta transfer integration test for falling back

* Minor improvements

* Fix typo

* Fix data race in WAL delta integration test, kill upsertions on count

* In WAL delta tests, compare all points, vectors and payload exactly

* add test for fallback in into straming if diff is big

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants