Strip deleted post ids from author deques in PostStore - #59
Open
Pitchfork-and-Torch wants to merge 1 commit into
Open
Strip deleted post ids from author deques in PostStore#59Pitchfork-and-Torch wants to merge 1 commit into
Pitchfork-and-Torch wants to merge 1 commit into
Conversation
mark_as_deleted dropped CompactPost and tombstoned the id, but left the id in the author's original/secondary/video deques. get_posts_from_map scans only the newest 500 TinyPosts, so a burst of deletes of recent posts could hide remaining live rows. Co-authored-by: Jon Bailey <Pitchfork-and-Torch@users.noreply.github.com>
cursor
Bot
force-pushed
the
cursor/thunder-delete-scan-window-2fbc
branch
from
August 15, 2026 04:31
1497592 to
74a8396
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
PostStore::mark_as_deletedremovedCompactPostand tombstoned the id, then only pushed aTinyPostontooriginal_posts_by_user[DELETE_EVENT_KEY=0]. It never pulled the id out of that author's original / secondary / video deques.TweetDeleteEventis{post_id, deleted_at}only, soauthor_idis lost onceCompactPostis dropped.get_posts_from_mapthen doesdeque.rev().take(MAX_TINY_POSTS_PER_USER_SCAN=500)andposts.get. Deleted ids sit at the back (newest), so they consume the 500-wide scan window. Age trim only pops the front.Repro: insert 600 originals for author A; delete the newest 500;
get_all_posts_by_users([A])returns[]even though 100 older live rows remain in the deque and inposts.This is not #48 (same-second insert drop) and not #57 (sibling-index overflow on insert). It is also not trim/GC, IPS, or the thrift/kafka/strato issues.
Fix
Look up
CompactPostviaposts.removeso we still haveauthor_id. Strip the id from that author's original / secondary / video deques. Keep thedeleted_posts+DELETE_EVENT_KEYtombstone so a later insert is still skipped. If the post was never inposts, tombstone-only.Tests
mark_as_deleted_does_not_starve_newest_scan_window: 600 video originals, delete newest 500, originals+videos still serve the remaining live rows.mark_as_deleted_strips_secondary_and_tombstones_unknown: reply deque is stripped; unknown id is tombstone-only; later insert of a deleted id is still skipped.Standalone harness (same insert/get/scan control flow): buggy path returns
[]; fixed path returns 50 live ids from the remaining 100.thunder/has no Cargo manifest in this snapshot, so the crate tests cannot run here.Lane:
thunder/posts/post_store.rsonly. Based onxai-org/x-algorithmmain (c65aa179).