findReplicaHavingCoveringPart may create a lot of excessive traffic

[filimonov]

Here we do full parts listing:

ClickHouse/src/Storages/StorageReplicatedMergeTree.cpp

Line 3168 in fea2836

Strings parts = zookeeper->getChildren(zookeeper_path + "/replicas/" + replica + "/parts");

If number of parts & replicas is big that creates significant amount of traffic between zookeeper and clickhouse.

I guess we can start with more 'point' lookup by exact part name cluster-wide.

[alex-zaitsev]

select event, formatReadableSize(value) 
 from system.events where event in ('InsertedBytes','ZooKeeperBytesSent','ZooKeeperBytesReceived');
┌─event──────────────────┬─formatReadableSize(value)─┐
│ InsertedBytes          │ 89.17 GiB                 │
│ ZooKeeperBytesSent     │ 111.04 GiB                │
│ ZooKeeperBytesReceived │ 13.14 TiB                 │
└────────────────────────┴───────────────────────────┘

[alex-zaitsev]

Up. Latest results. 140TB of ZK traffic on 500GB of inserts.

┌─event──────────────────┬─formatReadableSize(value)─┐
│ InsertedBytes          │ 515.58 GiB                │
│ ZooKeeperBytesSent     │ 542.53 GiB                │
│ ZooKeeperBytesReceived │ 142.56 TiB                │
└────────────────────────┴───────────────────────────┘