Company or project name
No response
Question
Hi team,
We are running a cluster of 6 nodes (2 shards, 3 replicas each) on v25.3.6.10034 altinitystable release with 3 zookeeper nodes as coordination layer.
Recently we wanted to add new rollup rule to test min and max functions.
Here is snippet of our retention.xml file before testing:
<?xml version="1.0"?>
<yandex>
<graphite_rollup>
<!-- general test rule -->
<pattern>
<regexp>^test\.</regexp>
<function>avg</function>
<retention>
<age>0</age>
<precision>60</precision>
</retention>
<retention>
<age>86400</age>
<precision>7200</precision>
</retention>
</pattern>
. . .
We tried to add additional rule above the one you have seen now, so file looked like this:
?xml version="1.0"?>
<yandex>
<graphite_rollup>
<!-- testing save of minimal metric -->
<pattern>
<regexp>^test\.minimal\.</regexp>
<function>min</function>
<retention>
<age>0</age>
<precision>60</precision>
</retention>
</pattern>
<!-- general test rule -->
<pattern>
<regexp>^test\.</regexp>
<function>avg</function>
<retention>
. . .
The issue is that after retention file is updated and clickhouse-server is stop-started, replica becomes read-only, throwing this error:
<Error> data.graphite (ReplicatedMergeTreeAttachThread): Initialization failed, table will remain readonly. Error: Code: 342. DB::Exception: Existing table metadata in ZooKeeper differs in graphite params. Stored in ZooKeeper hash: 314***350, local hash: 136***314. (METADATA_MISMATCH)...
We tried updating every node one-by-one, but whole cluster becomes read-only.
We also tried running SYSTEM RESTORE REPLICA and SYSTEM RESTART REPLICA commands, with no success.
Is this expected behaviour?
Or am I missing something and there is a way to change rollup without manipulations with zookeeper's metadata?
Company or project name
No response
Question
Hi team,
We are running a cluster of 6 nodes (2 shards, 3 replicas each) on v25.3.6.10034 altinitystable release with 3 zookeeper nodes as coordination layer.
Recently we wanted to add new rollup rule to test min and max functions.
Here is snippet of our retention.xml file before testing:
We tried to add additional rule above the one you have seen now, so file looked like this:
The issue is that after retention file is updated and clickhouse-server is stop-started, replica becomes read-only, throwing this error:
We tried updating every node one-by-one, but whole cluster becomes read-only.
We also tried running SYSTEM RESTORE REPLICA and SYSTEM RESTART REPLICA commands, with no success.
Is this expected behaviour?
Or am I missing something and there is a way to change rollup without manipulations with zookeeper's metadata?