Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

View update generator may process disowned tokens after aborted decommission #14392

Closed
bhalevy opened this issue Jun 26, 2023 · 3 comments
Closed
Assignees
Milestone

Comments

@bhalevy
Copy link
Member

bhalevy commented Jun 26, 2023

Decommission migrates tokens from the decommissioned node to the remaining nodes.
When a table as materialized view or secondary indexes, we generate view updates from the sstables containing those tokens that are put in the staging sub-directory.

When decommission is aborted, it is no longer necessary to generate those view updates and cleanup is run to remove those now-disowned tokens from the nodes, but cleanup skips the staging sstables due to the race with view building (see #9559).
It was observed that the view update generation process may take a long time, which holds up cleanup.
and even with #13812, we wait for view updates to be done before cleaning up the staging sstables.

This was addressed in #11932 by simply skipping over the disowned tokens in the view update generator reader, which may not be the optimal way, but it is correct and allows the view update generator to completely quickly after decommission is aborted, and the cleanup can run normally.

This issue is now opened for backporting purposes.

@bhalevy
Copy link
Member Author

bhalevy commented Jun 26, 2023

Fixed by 3aff59f

@DoronArazii
Copy link

@mykaul the milestone that has this patch (3aff59f) is 5.2
We need it backported to 5.1 if possible.
@scylladb/scylla-maint please assist.

@DoronArazii DoronArazii modified the milestones: 5.1, 5.2 Aug 8, 2023
@denesb
Copy link
Contributor

denesb commented Aug 9, 2023

Backported to 5.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants