Skip to content

fix buffered key below watermark#625

Merged
morgo merged 3 commits intoblock:mainfrom
morgo:fix-key-below-watermark
Feb 25, 2026
Merged

fix buffered key below watermark#625
morgo merged 3 commits intoblock:mainfrom
morgo:fix-key-below-watermark

Conversation

@morgo
Copy link
Collaborator

@morgo morgo commented Feb 25, 2026

A Pull Request should be associated with an Issue.

We wish to have discussions in Issues. A single issue may be targeted by multiple PRs.
If you're offering a new feature or fixing anything, we'd like to know beforehand in Issues,
and potentially we'll be able to point development in a particular direction.
Further notes in https://github.com/block/spirit/blob/main/.github/CONTRIBUTING.md

Fixes #624

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a critical logic bug in the bufferedMap subscription where the watermark optimization check was inverted. The bug caused keys below the low watermark (already copied by the copier) to be incorrectly skipped during flush operations, while keys at or above the watermark (not yet copied) were incorrectly flushed. This could lead to data race conditions, contention, and performance/memory issues from buffering too many versions.

Changes:

  • Fixed the watermark check condition in bufferedMap.Flush() by adding a negation operator to match deltaMap behavior
  • Updated test expectations in TestBufferedMapFlushWithoutLockRespectsWatermark to verify the correct behavior
  • Updated comments in the test to accurately describe the fixed watermark logic

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
pkg/repl/subscription_buffered.go Fixed inverted watermark logic by changing condition from KeyBelowLowWatermark(...) to !KeyBelowLowWatermark(...) and updated debug log message
pkg/repl/subscription_buffered_test.go Updated test expectations and comments to reflect the corrected watermark behavior, verifying that keys below the watermark are now flushed and keys at/above are skipped

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@morgo morgo requested a review from Kiran01bm February 25, 2026 19:23
Copy link
Collaborator

@Kiran01bm Kiran01bm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - also good that normal migrations (without buffering) was not impacted by this bug.

@morgo morgo merged commit 6ee6813 into block:main Feb 25, 2026
9 checks passed
@morgo morgo deleted the fix-key-below-watermark branch February 25, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logic for KeyBelowLowWatermark in subscription_buffered is incorrect

3 participants