to 3.0: fix: avoid duplicate rows in data branch diff replay#24141
Merged
heni02 merged 5 commits intomatrixorigin:3.0-devfrom Apr 15, 2026
Merged
Conversation
Treat objects deleted at or before the range end as transient intermediates during range replay regardless of appendable state, so delete-chain resolution follows the terminal successor instead of replaying both predecessor and successor objects. Add a regression test for the appendable-predecessor path on 3.0-dev. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
gouhongshen
changed the title
There was a problem hiding this comment.
Pull request overview
Ports a main-branch fix to 3.0-dev to prevent duplicate rows during DATA BRANCH DIFF range replay by ensuring delete-chain resolution returns only the terminal object visible at the range end, plus adds a regression test for the appendable-predecessor case.
Changes:
- Update delete-chain resolution to follow the successor for any object deleted at or before the replay range end (regardless of appendable state).
- Add a regression test intended to cover the “appendable predecessor deleted before end” scenario.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pkg/vm/engine/disttae/logtailreplay/change_handle.go | Adjusts delete-chain resolution to skip transient objects deleted by range end and follow successors to avoid duplicate replay. |
| pkg/vm/engine/disttae/logtailreplay/change_handle_test.go | Adds a new test intended to lock down the appendable-predecessor deletion case for delete-chain resolution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Write both the appendable predecessor and its successor into the in-memory file service so the 3.0-dev regression test exercises the pre-fix duplicate-replay path instead of passing via the missing-file rewrite path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Merge Queue Status
This pull request spent 1 hour 10 minutes 29 seconds in the queue, with no time running CI. Waiting for:
All conditions
ReasonPull request #24141 has been dequeued Pull request from fork cannot be queued. This pull request comes from a fork, and Mergify needs the author's permission to update its branch.
HintYou should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Which issue(s) this PR fixes:
issue #23751
What this PR does / why we need it:
This is the 3.0-dev port of the main-branch fix for duplicated DATA BRANCH DIFF replay rows.
Range replay could keep both an appendable predecessor and its TN successor when the predecessor had already been deleted at or before the replay end timestamp. After GC and restart, CollectChanges then replayed the same logical row from both objects, which surfaced as duplicated INSERT output such as a repeated 5,5 row.
This change makes delete-chain resolution treat any object deleted at or before the range end as a transient intermediate, regardless of appendable state, and follow its successor so the final replay set contains only the terminal object visible at the end timestamp.
A 3.0-dev regression test is added for the appendable-predecessor case to lock the behavior down.