Part XXX was found on disk default which is not defined in the storage policy

[zhlovy]

after i change the param: SET max_memory_usage = 20000000000;(Restart it before I know it doesn't need to be restarted) I restart the ck , but an exception happened :

---

<Error> ThreadPool: Exception in ThreadPool(max_threads: 32, max_free_threads: 32, queue_size: 32, shutdown_on_exception: 1).: Code: 479, e.displayText() = DB::Exception: Part 20200415_1566_2123_4was found on diskdefaultwhich is not defined in the storage policy: Cannot attach tabledefault.btrace from metadata file /home/q/clickhouse/data/metadata/default/btrace.sql from query ATTACH TABLE btrace (btraceIdString,platOpenIdString, appNameString,occTimeUInt64,occTimeDt DateTime, INDEX index_appName appName TYPE bloom_filter GRANULARITY 4) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/btrace', '{replica}') PARTITION BY toYYYYMMDD(occTimeDt) ORDER BY (occTime, platOpenId, appName, btraceId) TTL occTimeDt + toIntervalMonth(6) SETTINGS storage_policy = 'moving_from_ssd_to_hdd', index_granularity = 8192, Stack trace (when copying this message, always include the lines below):

my ck is 20.3.4.10

I move this part to detached is useless ,it will report another part is same problem;

In order to restart the ck on this machine, I had to delete all the data so that the data was synchronized from another replica machine
I don't want to do this every time when I restart the service

Can someone please take a look, thanks

<Error> ThreadPool: Exception in ThreadPool(max_threads: 32, max_free_threads: 32, queue_size: 32, shutdown_on_exception: 1).: Code: 479, e.displayText() = DB::Exception: Part `20200415_1566_2123_4` was found on disk `default` which is not defined in the storage policy: Cannot attach table `default`.`btrace` from metadata file /home/q/clickhouse/data/metadata/default/btrace.sql from query ATTACH TABLE btrace (`btraceId` String, `platOpenId` String,  `appName` String, `occTime` UInt64, `occTimeDt` DateTime, INDEX index_appName appName TYPE bloom_filter GRANULARITY 4) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/btrace', '{replica}') PARTITION BY toYYYYMMDD(occTimeDt) ORDER BY (occTime, platOpenId, appName, btraceId) TTL occTimeDt + toIntervalMonth(6) SETTINGS storage_policy = 'moving_from_ssd_to_hdd', index_granularity = 8192, Stack trace (when copying this message, always include the lines below):
0. Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x102e0d8c in /usr/bin/clickhouse
1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x8f2d989 in /usr/bin/clickhouse
2. ? @ 0xd7647cd in /usr/bin/clickhouse
3. DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, DB::StorageID const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::StorageInMemoryMetadata const&, DB::Context&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::MergeTreeData::MergingParams const&, std::__1::unique_ptr<DB::MergeTreeSettings, std::__1::default_delete<DB::MergeTreeSettings> >, bool) @ 0xd5d98a4 in /usr/bin/clickhouse
4. ? @ 0xd9585da in /usr/bin/clickhouse
5. std::__1::__function::__func<std::__1::shared_ptr<DB::IStorage> (*)(DB::StorageFactory::Arguments const&), std::__1::allocator<std::__1::shared_ptr<DB::IStorage> (*)(DB::StorageFactory::Arguments const&)>, std::__1::shared_ptr<DB::IStorage> (DB::StorageFactory::Arguments const&)>::operator()(DB::StorageFactory::Arguments const&) @ 0xd95baf3 in /usr/bin/clickhouse
6. DB::StorageFactory::get(DB::ASTCreateQuery const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, DB::Context&, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, bool) const @ 0xd4fc66c in /usr/bin/clickhouse
7. DB::createTableFromAST(DB::ASTCreateQuery, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool) @ 0xcedc88e in /usr/bin/clickhouse
8. ? @ 0xced33bf in /usr/bin/clickhouse
9. ? @ 0xced3b71 in /usr/bin/clickhouse
10. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0x8f51c4b in /usr/bin/clickhouse
11. ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() const @ 0x8f528c4 in /usr/bin/clickhouse
12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x8f50b4b in /usr/bin/clickhouse
13. ? @ 0x8f4f00f in /usr/bin/clickhouse
14. start_thread @ 0x7dd5 in /usr/lib64/libpthread-2.17.so
15. clone @ 0xfdead in /usr/lib64/libc-2.17.so
 (version 20.3.4.10 (official build))

[filimonov]

The check is intended to prevent accidental loss of some fragments of the table during reconfiguration.

The main question here is why the part for the table default.btrace appeared on the disk which is not listed in moving_from_ssd_to_hdd storage policy.

Normally that should not happen and may be caused by some misconfiguration or by inaccureate table movements.

[zhlovy]

yes
Parameter of 'max_memory_usage' changes should not affect the restart of the service
so this maybe a bug or
do we need to pay attentions to restart the server when storage policy is 'moving_from_ssd_to_hdd' ?

[alexey-milovidov]

It may happen if you edited the storage policy in configuration and removed/changed some of the disks.
Then some parts may remain on disk that no longer belongs to the storage policy of the table.

[bcxu]

@zhlovy I encountered the same problem as you and reported this error after restart. How did you solve this problem later? I didn't find the corresponding solution on other websites. Thank you!