Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport to 22.8: Fix moving broken parts to the detached for the object storage disk on startup #48273

Conversation

aalexfvk
Copy link
Contributor

@aalexfvk aalexfvk commented Mar 31, 2023

Clickhouse-server version 22.8 throws exception on startup when a metadata file on object storage disk has incorrect content.
The fixing code has been extracted from the PR

Error log:

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xa3fb75a in /usr/bin/clickhouse
1. DB::throwReadAfterEOF() @ 0xa4026c0 in /usr/bin/clickhouse
2. void DB::readIntTextImpl<unsigned int, void, (DB::ReadIntTextCheckOverflow)0>(unsigned int&, DB::ReadBuffer&) @ 0xa5af217 in /usr/bin/clickhouse
3. DB::DiskObjectStorageMetadata::deserialize(DB::ReadBuffer&) @ 0x145994e3 in /usr/bin/clickhouse
4. DB::MetadataStorageFromDisk::readMetadataUnlocked(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_lock<std::__1::shared_mutex>&) const @ 0x14594878 in /usr/bin/clickhouse
5. DB::MetadataStorageFromDisk::getFileSize(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x14594446 in /usr/bin/clickhouse
6. DB::DiskRestartProxy::getFileSize(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x145e4f11 in /usr/bin/clickhouse
7. ? @ 0x157a8dcd in /usr/bin/clickhouse
8. DB::DataPartStorageOnDisk::calculateTotalSizeOnDisk() const @ 0x157a8938 in /usr/bin/clickhouse
9. ? @ 0x158e491d in /usr/bin/clickhouse
10. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0xa4bf9e6 in /usr/bin/clickhouse
11. void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()>(void&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0xa4c1337 in /usr/bin/clickhouse
12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xa4bd308 in /usr/bin/clickhouse
13. ? @ 0xa4c051d in /usr/bin/clickhouse
14. start_thread @ 0x76db in /lib/x86_64-linux-gnu/libpthread-2.27.so
15. __clone @ 0x12161f in /lib/x86_64-linux-gnu/libc-2.27.so
 (version 22.8.13.20 (official build))

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Not for changelog (changelog entry is not required)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

@aalexfvk aalexfvk changed the title Fix loading broken parts for the object storage disk Fix moving broken parts to the detached for the object storage disk on startup Mar 31, 2023
@alexey-milovidov alexey-milovidov changed the title Fix moving broken parts to the detached for the object storage disk on startup Backport to 22.8: Fix moving broken parts to the detached for the object storage disk on startup Mar 31, 2023
@alexey-milovidov alexey-milovidov added pr-backport Changes, backported to release branch. Do not use manually - automated use only! can be tested Allows running workflows for external contributors labels Mar 31, 2023
@alexey-milovidov alexey-milovidov merged commit 93a89b4 into ClickHouse:22.8 Mar 31, 2023
1 check passed
@alexey-milovidov alexey-milovidov self-assigned this Mar 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
can be tested Allows running workflows for external contributors pr-backport Changes, backported to release branch. Do not use manually - automated use only!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants