replication: do not fetch records twice in a full mesh #3294

kostja · 2018-03-27T19:22:37Z

Extend IPROTO_SUBSCRIBE command with a list of server ids for which SUBSCRIBE should fetch changes. In a full mesh configuration, only download records originating from the immediate peer. Do not download the records from other peers twice.

For example, imagine a full mesh of 3 replicas. Currently each Tarantool instance will download from all peers all recorrds in their WAL excepts records with instance id equal to the self instance id. Instead, it could send a subscribe request to its peers with server ids which are not present in other subscribe requests.

Implementaiton

After issuing IPROTO_REQUEST_VOTE to all peers we know a map of server ids, their peers and their vclocks. Sort the map by server id. Iterate over each server in the list of peers, and assign its id to this server's SUBSCRIBE request. Assign all the remaining ids in _cluster table to the last peer (alternatively, if there are many ids in the remainder, keep going through the list of server and assign "orphan" ids in round-robin fashion).
Issue the subscribe request.

After this feature is implemented, each time a server responsible feeding more than 1 server id is dropped, we need to re-subscribe to some other peer and reassign the dropped ids to that peer. Each time a server is connected again, we need to rebalance again.

Extend IPROTO_SUBSCRIBE command with a list of server ids for which SUBSCRIBE should fetch changes. In a full mesh configuration, only download records originating from the immediate peer. Do not download the records from other peers twice. Proposal for #3294

In a full mesh configuration, only download records originating from the immediate peer. Do not download the records from other peers twice. Extend IPROTO_SUBSCRIBE command with a list of server UUIDs for which SUBSCRIBE should fetch changes. Implemet subscribe daemon, a Lua script to dinamically reassign appliers with a list of UUIDs to fetch changes from. Proposal for #3294

Extend IPROTO_SUBSCRIBE with a list of UUIDs to fetch changes from. Implement src_black_list in relay to filter out rows originating from replicas in this list. This is required for #3294

In a full mesh configuration, only download records originating from the immediate peer. Do not download the records from other peers twice. Extend IPROTO_SUBSCRIBE command with a list of server UUIDs for which SUBSCRIBE should fetch changes. Implemet subscribe daemon, a Lua script to dinamically reassign appliers with a list of UUIDs to fetch changes from. Proposal for #3294

Extend IPROTO_SUBSCRIBE with a list of UUIDs to fetch changes from. Implement src_black_list in relay to filter out rows originating from replicas in this list. This is required for #3294

In a full mesh configuration, only download records originating from the immediate peer. Do not download the records from other peers twice. Extend IPROTO_SUBSCRIBE command with a list of server UUIDs for which SUBSCRIBE should fetch changes. Implemet subscribe daemon, a Lua script to dinamically reassign appliers with a list of UUIDs to fetch changes from. Proposal for #3294

Extend IPROTO_SUBSCRIBE with a list of UUIDs to fetch changes from. Implement src_black_list in relay to filter out rows originating from replicas in this list. This is required for #3294

This is a draft paper covering follwing topics: 1. Draft protocol of discovering and maintaining network topology in case of large arbitrary network. 2. Required changes in relay and applier to support this feature. 3. Introducing the subscription daemon - a Lua sript implementing protocol logic and responsible for network consistency maintaining. Needed for #3294

This is a draft paper covering following topics: 1. Draft protocol for discovering and maintaining network topology in case of large arbitrary network. 2. List of required changes to support this feature. 3. Open questions and alternatives. RFC for #3294

This is a draft paper covering following topics: 1. Draft protocol for discovering and maintaining network topology in case of large arbitrary network. 2. List of required changes to support this feature. 3. Open questions and alternatives. Changes in V2: Based or Vlad's review 1. Rewrite couple sections to make it more clear. 2. Clarify with more details and add examples. 3. Fixed error. RFC for #3294

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_MASK. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key Add key `IPROTO_ID_MASK = 0x51` to the internals reference.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_MASK. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_MASK = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_MASK | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty.

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty. (cherry picked from commit 45de990)

@TarantoolBot

Add a filter for relay to skip rows coming from unwanted instances. A list of instance ids whose rows replica doesn't want to fetch is encoded together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER. Filtering rows is needed to prevent an instance from fetching its own rows from a remote master, which is useful on initial configuration and harmful on resubscribe. Prerequisite #4739, #3294 @TarantoolBot document Title: document new binary protocol key and subscribe request changes Add key `IPROTO_ID_FILTER = 0x51` to the internals reference. This is an optional key used in SUBSCRIBE request followed by an array of ids of instances whose rows won't be relayed to the replica. SUBSCRIBE request is supplemented with an optional field of the following structure: ``` +====================+ | ID_FILTER | | 0x51 : ID LIST | | MP_INT : MP_ARRRAY | | | +====================+ ``` The field is encoded only when the id list is not empty. (cherry picked from commit 45de990)

kostja added feature A new functionality replication labels Mar 27, 2018

kostja added this to the 1.10.1 milestone Mar 27, 2018

kostja assigned kbelyavs Mar 27, 2018

kbelyavs added the design review label Jul 23, 2018

kyukhin added the tmp label Aug 2, 2018

kyukhin modified the milestones: 1.10.2, 1.10.3 Aug 3, 2018

kyukhin removed the tmp label Aug 20, 2018

kyukhin modified the milestones: 1.10.3, 2.1.1 Oct 25, 2018

kostja modified the milestones: 2.1.1, 2.2.0 Nov 19, 2018

kostja unassigned kbelyavs Nov 19, 2018

kyukhin removed this from the 2.2.0 milestone Jun 6, 2019

kyukhin added this to the 2.3.0 milestone Jun 6, 2019

kyukhin modified the milestones: 2.3.1, 2.4.1 Nov 21, 2019

kyukhin modified the milestones: 2.4.1, 2.5.1 Jan 23, 2020

kyukhin modified the milestones: 2.5.1, 2.6.1 Apr 10, 2020

kyukhin modified the milestones: 2.6.1, wishlist Oct 23, 2020

Mons added backlog and removed design review labels Dec 4, 2020

kyukhin removed the backlog label Oct 15, 2021

sergepetrenko added the 13sp label Dec 17, 2021

TarantoolBot removed the 13sp label Jun 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

replication: do not fetch records twice in a full mesh #3294

replication: do not fetch records twice in a full mesh #3294

kostja commented Mar 27, 2018

replication: do not fetch records twice in a full mesh #3294

replication: do not fetch records twice in a full mesh #3294

Comments

kostja commented Mar 27, 2018

Implementaiton