-
Notifications
You must be signed in to change notification settings - Fork 852
storcon: Introduce deletion tombstones to support flaky node scenario #12096
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
dca4d59 to
b87860e
Compare
8459 tests run: 7877 passed, 0 failed, 582 skipped (full report)Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
cee59e7 at 2025-06-06T06:00:23.378Z :recycle: |
b87860e to
348dc4c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks good. Got some points around robustness. Let me know if you wanna walk through anything on a call.
Let's add a test that covers our main scenario: preventing a flaky node from re-attaching by marking it soft deleted. Bonus points for covering concurrency between re-attach and node deletion. test_storage_controller.py is a good starting point.
Please update the cover letter with a description of the problem we are solving here.
It's better to have actual descriptions in the commit history than GH links.
1ae2b74 to
17926bf
Compare
|
If this PR added a GUC in the Postgres fork or If you're an external contributor, a Neon employee will assist in |
17926bf to
7deb448
Compare
VladLazar
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - give it a go in staging and update the docs for PS decom if required.
Small piece of feedback: we usually resolve review comments with a link to the commit that fixed it. This makes it easier for the reviewer to know if the change was applied and where.
…#12096) ## Problem Removed nodes can re-add themselves on restart if not properly tombstoned. We need a mechanism (e.g. soft-delete flag) to prevent this, especially in cases where the node is unreachable. More details there: #12036 ## Summary of changes - Introduced `NodeLifecycle` enum to represent node lifecycle states. - Added a string representation of `NodeLifecycle` to the `nodes` table. - Implemented node removal using a tombstone mechanism. - Introduced `/debug/v1/tombstone*` handlers to manage the tombstone state.
…#12096) ## Problem Removed nodes can re-add themselves on restart if not properly tombstoned. We need a mechanism (e.g. soft-delete flag) to prevent this, especially in cases where the node is unreachable. More details there: #12036 ## Summary of changes - Introduced `NodeLifecycle` enum to represent node lifecycle states. - Added a string representation of `NodeLifecycle` to the `nodes` table. - Implemented node removal using a tombstone mechanism. - Introduced `/debug/v1/tombstone*` handlers to manage the tombstone state.
Problem
Removed nodes can re-add themselves on restart if not properly tombstoned. We need a mechanism (e.g. soft-delete flag) to prevent this, especially in cases where the node is unreachable.
More details there: #12036
Summary of changes
NodeLifecycleenum to represent node lifecycle states.NodeLifecycleto thenodestable./debug/v1/tombstone*handlers to manage the tombstone state.