Not to allow create different table with same zookeeper replica path

[yx91490]

Use case

I using version 20.8.11.17.lts. One day I created a table db2.tab1 with same zookeeper replica path as table db1.tab1, then I found some errors in log:

<Error> executeQuery: Code: 220, e.displayText() = DB::Exception: Duplicate interserver IO endpoint: DataPartsExchange:/clickhouse/tables/01/db2.tab1/replicas/xxxx (version 20.8.11.17 (official build)) (from 0.0.0.0:0)

then I restart clickhouse instance but cannot start successfully, Similar error as above :

2021.04.08 03:10:11.605796 [ 4093 ] {} <Error> Application: Caught exception while loading metadata: Code: 231, e.displayText() = DB::Exception: The local set of parts of table db1.tab1 doesn't look like the set of parts in ZooKeeper: 244.00 rows of 244.00 total rows in filesystem are suspicious. There are 6 unexpected parts with 244 rows (0 of them is not just-written with 244 rows), 0 missing parts (with 0 blocks).: Cannot attach table `db1`.`tab1` from metadata file /apps/data/clickhouse-1/metadata/db1/tab1.sql from query ATTACH TABLE db1.tab1 (`department` String, `city` String, `begin_month` String , `transfer_ratio` Decimal(18, 2) COMMENT, `pre_ratio` Decimal(18, 2), `city_transfer_ratio` Decimal(18, 2), `city_pre_ratio` Decimal(18, 2), `dt` String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/db2.tab1', '{replica}') PARTITION BY dt ORDER BY (department, city) SETTINGS index_granularity = 8192: while loading database `db1` from path /apps/data/clickhouse-1/metadata/db1, Stack trace (when copying this message, always include the lines below)
...
2021.04.08 03:11:50.741636 [ 4508 ] {} <Error> Application: Caught exception while loading metadata: Code: 220, e.displayText() = DB::Exception: Duplicate interserver IO endpoint: DataPartsExchange:/clickhouse/tables/01/db2.tab1/replicas/xxx: while loading database `db2` from path /apps/data/clickhouse-1/metadata/db2, Stack trace (when copying this message, always include the lines below):

until I delete db1.tab1 metadata in disk I can start clickhouse instance successfully.

Describe the solution you'd like

So I suggest Not to allow create different table with same zookeeper replica path to avoid this mistake error.

Describe alternatives you've considered

No

Additional context

No

[tavplubix]

It's not allowed to create different tables with the same zookeeper path and replica name, such CREATE query should fail with Replica already exists error. Could you please provide more information about how did you create the second table? Do you use auxiliary ZooKeepers configs?

[alexey-milovidov]

No answer.

[tavplubix]

Seems like it's possible to reproduce Duplicate interserver IO endpoint if create two replicated tables with the same zookeeper_path and replica_name, but in different ZooKeepers (if auxiliary ZooKeepers are configured)

[yx91490]

It's not allowed to create different tables with the same zookeeper path and replica name, such CREATE query should fail with Replica already exists error. Could you please provide more information about how did you create the second table? Do you use auxiliary ZooKeepers configs?

Sorry for long time no reply, there is no < auxiliary_zookeepers> found in config file within /etc/clickhouse-server.

[yx91490]

It's not allowed to create different tables with the same zookeeper path and replica name, such CREATE query should fail with Replica already exists error. Could you please provide more information about how did you create the second table? Do you use auxiliary ZooKeepers configs?

I remembered that I remove the replica path in zookeeper when I first create table db2.tab1 failed for 'replica already exists'， then I create successfully. It is not a regular operation😂

[yx91490]

How about check unique replica path when insert recoud to system.tables?

[alexey-milovidov]

I remove the replica path in zookeeper

This is the culprit. We don't expect manual intervention in ZooKeeper.

[cyberhuman]

Seems like it's possible to reproduce Duplicate interserver IO endpoint if create two replicated tables with the same zookeeper_path and replica_name, but in different ZooKeepers (if auxiliary ZooKeepers are configured)

We have this issue. There are several auxiliary ZK and CH keepers configured and it's not possible to re-use the same path because of the error above. No manual editing of data in ZK/CH keeper was involved.