streaming: stale entries in system_distributed.view_build_status causes unnecessary view building during streaming #11905
Comments
denesb
added a commit
to denesb/scylla
that referenced
this issue
Nov 7, 2022
… non-member hosts We currently don't clean up the system_distributed.view_build_status table after removed nodes. This can cause false-positive check for whether view update generation is needed for streaming. The proper fix is to clean up this table, but that will be more involved, it even when done, it might not be immediate. So until then and to be on the safe side, filter out entries belonging to unknown hosts from said table. Fixes: scylladb#11905 Refs: scylladb#11836
syuu1228
pushed a commit
to syuu1228/scylla
that referenced
this issue
Jan 30, 2023
… non-member hosts We currently don't clean up the system_distributed.view_build_status table after removed nodes. This can cause false-positive check for whether view update generation is needed for streaming. The proper fix is to clean up this table, but that will be more involved, it even when done, it might not be immediate. So until then and to be on the safe side, filter out entries belonging to unknown hosts from said table. Fixes: scylladb#11905 Refs: scylladb#11836 Closes scylladb#11860
|
@denesb I think this merits a backport |
denesb
added a commit
that referenced
this issue
Mar 22, 2023
… non-member hosts We currently don't clean up the system_distributed.view_build_status table after removed nodes. This can cause false-positive check for whether view update generation is needed for streaming. The proper fix is to clean up this table, but that will be more involved, it even when done, it might not be immediate. So until then and to be on the safe side, filter out entries belonging to unknown hosts from said table. Fixes: #11905 Refs: #11836 Closes #11860 (cherry picked from commit 84a69b6)
denesb
added a commit
that referenced
this issue
Mar 22, 2023
… non-member hosts We currently don't clean up the system_distributed.view_build_status table after removed nodes. This can cause false-positive check for whether view update generation is needed for streaming. The proper fix is to clean up this table, but that will be more involved, it even when done, it might not be immediate. So until then and to be on the safe side, filter out entries belonging to unknown hosts from said table. Fixes: #11905 Refs: #11836 Closes #11860 (cherry picked from commit 84a69b6)
denesb
added a commit
that referenced
this issue
Mar 22, 2023
… non-member hosts We currently don't clean up the system_distributed.view_build_status table after removed nodes. This can cause false-positive check for whether view update generation is needed for streaming. The proper fix is to clean up this table, but that will be more involved, it even when done, it might not be immediate. So until then and to be on the safe side, filter out entries belonging to unknown hosts from said table. Fixes: #11905 Refs: #11836 Closes #11860 (cherry picked from commit 84a69b6)
|
Backported to 5.2, 5.1, 5.0 and 2021.1. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When streaming, said table is scanned for any entries with
status=STARTED. Due to #11836, this table can contain stale entries for nodes that already left the cluster. Currently these entries are also taken into consideration, possibly causing a false-positive conclusion that views are in the process of being built and therefore that streaming has to do view building itself inflicting unnecessary extra load on replicas at the worst possible time.The text was updated successfully, but these errors were encountered: