New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Small improvements for tables on write-once (s3_plain)/read-only (web) disks #59170
Commits on Jan 26, 2024
-
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Do not drop parts for static (ro/write-once) storages for Replicated …
…engine Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Add a note for MetadataStorageFromPlainObjectStorage to support trans…
…actions Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Disable mutable operations into {Replicated,}MergeTree with static st…
…orage Mutable operations includes: - INSERT - OPTIMIZE - parts manipulation - ... But does not include DROP, since you need an ability to DROP it. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Implement MetadataStorageFromPlainObjectStorageTransaction::removeDir…
…ectory (for DROP TABLE) This is required to remove format_version.txt for MergeTree after DROP (lldb) bt * thread 192, name = 'BgSchPool', stop reason = breakpoint 1.1 frame 0: 0x0000000016c7f452 clickhouse`__cxa_throw(thrown_object=0x00007ffd7a53c480, tinfo=0x00000000011602a0, dest=(clickhouse`DB::Exception::~Exception() at Exception.h:40)) + 18 at cxa_exception.cpp:258 frame 1: 0x00000000109bf709 clickhouse`DB::IMetadataTransaction::throwNotImplemented() + 137 at IMetadataStorage.h:151 frame 2: 0x00000000109bf126 clickhouse`DB::IMetadataTransaction::removeDirectory(this=<unavailable>, (null)=<unavailable>) + 6 at IMetadataStorage.h:96 frame 3: 0x0000000010968978 clickhouse`DB::(anonymous namespace)::RemoveRecursiveObjectStorageOperation::removeMetadataRecursive(this=0x00007ffd7a7327a0, tx=std::__1::shared_ptr<DB::IMetadataTransaction>::element_type @ 0x00007ffd7a6f6018 strong=4 weak=1, path_to_remove="tmp_insert_all_2_2_0") + 376 at DiskObjectStorageTransaction.cpp:364 frame 4: 0x000000001096893d clickhouse`DB::(anonymous namespace)::RemoveRecursiveObjectStorageOperation::removeMetadataRecursive(this=0x00007ffd7a7327a0, tx=std::__1::shared_ptr<DB::IMetadataTransaction>::element_type @ 0x00007ffd7a6f6018 strong=4 weak=1, path_to_remove="store/27f/27f6f9d2-ab2e-4817-b771-0e706e485b15/") + 317 at DiskObjectStorageTransaction.cpp:362 frame 5: 0x0000000010965a7f clickhouse`DB::(anonymous namespace)::RemoveRecursiveObjectStorageOperation::execute(this=0x00007ffd7a7327a0, tx=std::__1::shared_ptr<DB::IMetadataTransaction>::element_type @ 0x00007ffd7a6f6018 strong=4 weak=1) + 95 at DiskObjectStorageTransaction.cpp:372 frame 6: 0x0000000010963a2f clickhouse`DB::DiskObjectStorageTransaction::commit(this=0x00007ffea22b6168) + 79 at DiskObjectStorageTransaction.cpp:903 frame 7: 0x00000000109577e5 clickhouse`DB::DiskObjectStorage::removeSharedRecursive(this=<unavailable>, path="store/27f/27f6f9d2-ab2e-4817-b771-0e706e485b15/", keep_all_batch_data=false, file_names_remove_metadata_only=size=0) + 133 at DiskObjectStorage.cpp:426 frame 8: 0x000000001095af79 clickhouse`DB::DiskObjectStorage::removeRecursive(this=<unavailable>, path=<unavailable>) + 57 at DiskObjectStorage.h:79 * frame 9: 0x0000000010b5c3d5 clickhouse`DB::DatabaseCatalog::dropTableFinally(this=0x00007ffff7096800, table=0x00007ffe853e1bd0) + 1365 at DatabaseCatalog.cpp:1317 frame 10: 0x0000000010b5b7e7 clickhouse`DB::DatabaseCatalog::dropTableDataTask(this=0x00007ffff7096800) + 2375 at DatabaseCatalog.cpp:1270 Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Fix MetadataStorageFromPlainObjectStorage::iterateDirectory() (add pr…
…efix) Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
tests: cover DROP TABLE on s3_plain disk
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Prohibit DROP from static storages
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Prohibit CREATE on static storages (ATTACH should be used instead)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
tests: rewrite test for MergeTree engines in s3_plain using ATTACH
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Revert "Prohibit DROP from static storages"
It is actually OK to DROP at least from read-only disk, as for the write-once it should be OK as well, since CREATE is prohibited anyway, so there should be no leftovers CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/stress_test__asan_.html CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/integration_tests__asan__[3_4].html This reverts commit 082e627. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
tests/test_disk_over_web_server: update expected assertion for static…
… storage Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Do not create format_version.txt on write-once storages
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
Revert "Prohibit CREATE on static storages (ATTACH should be used ins…
…tead)" CREATE is used by Replicated database even for ATTACH, and anyway ATTACH creates format_version.txt as well (without previuos patch). This reverts commit 323f1f320ebe7e588d443abf78efa2c60193e7a9. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
-
tests: adjust 02980_s3_plain_DROP_TABLE tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Commits on Jan 27, 2024
-
tests: fix 02980_s3_plain_DROP_TABLE_ReplicatedMergeTree flakiness
In case of commit failed (likely due to fault injection in tests) part name will have different name, not all_0_0_0, so replace numbers with X. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>