Replace topology::transition_state::publish_cdc_generation
with a queue of generations to publish and bg fiber to do it
#15194
Milestone
Currently topology coordinator has a
topology::transition_state::publish_cdc_generation
state where already created cdc generation is published to the user visible tables. This process cannot fail even if the bootstrap is aborted (otherwise some cdc updates will be missed)! With gossiper we have a fiber on each node that tries to publish the latest generation until it eventually succeed. With topology coordinator we have two options: either we hang inpublish_cdc_generation
state until it succeeds (this will block all other topology operations) or replacepublish_cdc_generation
with a static row in the topology state table that will hold not yet published generations. Instead of moving topublish_cdc_generation
the generation will be added to the row. The topology coordinator will start a background fiber that will constantly try to publish all not yet published cdc generations atomically.To publish cdc generation atomically
system_distributed_keyspace::create_cdc_desc
need to be changed to handle multiple generations and write multiple timestamps intocdc_generation_timestamps
atomically.The text was updated successfully, but these errors were encountered: