Skip to content

Revert "Revert "Add a test for #56334""#84143

Merged
alexey-milovidov merged 11 commits intomasterfrom
revert-84024-revert-84016-block-structure-mismatch-after-metadata-only-alter
Jan 26, 2026
Merged

Revert "Revert "Add a test for #56334""#84143
alexey-milovidov merged 11 commits intomasterfrom
revert-84024-revert-84016-block-structure-mismatch-after-metadata-only-alter

Conversation

@alexey-milovidov
Copy link
Member

@alexey-milovidov alexey-milovidov commented Jul 21, 2025

Reverts #84024

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

After metadata-only ALTERs, such as extending the elements of Enums, the optimization of aggregation with projection may end up producing an exception.

@clickhouse-gh
Copy link
Contributor

clickhouse-gh bot commented Jul 21, 2025

Workflow [PR], commit [5135aee]

Summary:

@clickhouse-gh clickhouse-gh bot added the pr-not-for-changelog This PR should not be mentioned in the changelog label Jul 21, 2025
@devcrafter
Copy link
Member

devcrafter commented Oct 21, 2025

@alexey-milovidov 03376_bloom_filter_has_const_array which failed before with PR was removed. Currently, this has only failure related to the introduced test 03571_block_structure_mismatch_after_metadata_only_alter.sql (w/o PR)

2961376:2025.10.21 00:19:12.726428 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Debug> executeQuery: (from [::1]:51476) (comment: f03571_block_structure_mismatch_after_metadata_only_alter.sql-test_9kxny9f9) (query 5, line 20) SELECT product FROM t GROUP BY product ORDER BY product; (stage: Complete)
2961377:2025.10.21 00:19:12.726786 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2961378:2025.10.21 00:19:12.726820 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Trace> Aggregator: Adjusting memory limit before external aggregation with 5.95 GiB (ratio: 0.2, available system memory: 29.75 GiB)
2961380:2025.10.21 00:19:12.727436 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Fatal> : Logical error: 'Block structure mismatch in AggregatingStep stream: different types:
2961484:2025.10.21 00:19:12.753943 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Fatal> : Stack trace (when copying this message, always include the lines below):
2961380:2025.10.21 00:19:12.727436 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Fatal> : Logical error: 'Block structure mismatch in AggregatingStep stream: different types:
2961484:2025.10.21 00:19:12.753943 [ 289866 ] {957d79f9-9ff5-41f2-8048-d14fb480eab9} <Fatal> : Stack trace (when copying this message, always include the lines below):
2961511:2025.10.21 00:19:12.760279 [ 1644 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
2961512:2025.10.21 00:19:12.760302 [ 1644 ] {} <Fatal> BaseDaemon: (version 25.10.1.1 (official build), build id: 788FF3D451F29CA2954F5FAF6AB7FD1A0B2F97B0, git hash: edefb984236c45db0f5fd8e2f984ecbe5835e438, architecture: x86_64) (from thread 289866) Received signal 6
2961513:2025.10.21 00:19:12.760330 [ 1644 ] {} <Fatal> BaseDaemon: Signal description: Aborted
2961514:2025.10.21 00:19:12.760335 [ 1644 ] {} <Fatal> BaseDaemon:
2961515:2025.10.21 00:19:12.760346 [ 1644 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f7eff7049fd 0x00007f7eff6b0476 0x00007f7eff6967f3 0x000055cde8647d29 0x000055cde86485ac 0x000055cde864886c 0x000055cde144f50e 0x000055cde144f100 0x000055cdecd40c2b 0x000055cdecd34a76 0x000055cdecd3ba73 0x000055cdf1f64690 0x000055cdf2151422 0x000055cdf20e63
f9 0x000055cdf1ffb20a 0x000055cdf1ffa742 0x000055cdee2079df 0x000055cdee5559c7 0x000055cdee550f92 0x000055cdf1a678ca 0x000055cdf1a84dd6 0x000055cdf6a72247 0x000055cdf6a727fe 0x000055cdf6a119bf 0x000055cdf6a0efcf 0x00007f7eff702ac3 0x00007f7eff794850
2961516:2025.10.21 00:19:12.760354 [ 1644 ] {} <Fatal> BaseDaemon: ########################################
2961517:2025.10.21 00:19:12.760383 [ 1644 ] {} <Fatal> BaseDaemon: (version 25.10.1.1 (official build), build id: 788FF3D451F29CA2954F5FAF6AB7FD1A0B2F97B0, git hash: edefb984236c45db0f5fd8e2f984ecbe5835e438) (from thread 289866) (query_id: 957d79f9-9ff5-41f2-8048-d14fb480eab9) (query: SELECT product
2961521:2025.10.21 00:19:12.760402 [ 1644 ] {} <Fatal> BaseDaemon:
2961522:2025.10.21 00:19:12.760413 [ 1644 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f7eff7049fd 0x00007f7eff6b0476 0x00007f7eff6967f3 0x000055cde8647d29 0x000055cde86485ac 0x000055cde864886c 0x000055cde144f50e 0x000055cde144f100 0x000055cdecd40c2b 0x000055cdecd34a76 0x000055cdecd3ba73 0x000055cdf1f64690 0x000055cdf2151422 0x000055cdf20e63
f9 0x000055cdf1ffb20a 0x000055cdf1ffa742 0x000055cdee2079df 0x000055cdee5559c7 0x000055cdee550f92 0x000055cdf1a678ca 0x000055cdf1a84dd6 0x000055cdf6a72247 0x000055cdf6a727fe 0x000055cdf6a119bf 0x000055cdf6a0efcf 0x00007f7eff702ac3 0x00007f7eff794850
2961526:2025.10.21 00:19:12.760470 [ 1644 ] {} <Fatal> BaseDaemon: 3. ? @ 0x00000000000969fd
2961528:2025.10.21 00:19:12.760483 [ 1644 ] {} <Fatal> BaseDaemon: 4. ? @ 0x0000000000042476
2961530:2025.10.21 00:19:12.760504 [ 1644 ] {} <Fatal> BaseDaemon: 5. ? @ 0x00000000000287f3
2961547:2025.10.21 00:19:12.770237 [ 1644 ] {} <Fatal> BaseDaemon: 6. ./ci/tmp/build/./src/Common/Exception.cpp:52: DB::abortOnFailedAssertion(String const&, void* const*, unsigned long, unsigned long) @ 0x0000000010559d29
2961637:2025.10.21 00:19:12.807274 [ 1644 ] {} <Fatal> BaseDaemon: 7. ./ci/tmp/build/./src/Common/Exception.cpp:85: DB::handle_error_code(String const&, std::basic_string_view<char, std::char_traits<char>>, int, bool, std::vector<void*, std::allocator<void*>> const&) @ 0x000000001055a5ac
2961705:2025.10.21 00:19:12.827751 [ 1644 ] {} <Fatal> BaseDaemon: 8. ./ci/tmp/build/./src/Common/Exception.cpp:136: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001055a86c
2961722:2025.10.21 00:19:12.874759 [ 1644 ] {} <Fatal> BaseDaemon: 9. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000936150e
2961724:2025.10.21 00:19:12.918296 [ 1644 ] {} <Fatal> BaseDaemon: 10. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000009361100
2961783:2025.10.21 00:19:12.936092 [ 1644 ] {} <Fatal> BaseDaemon: 11. ./src/Common/Exception.h:141: DB::Exception::Exception<std::basic_string_view<char, std::char_traits<char>>&, String, String>(int, FormatStringHelperImpl<std::type_identity<std::basic_string_view<char, std::char_traits<char>>&>::type, std::type_identity<String>::type, std
::type_identity<String>::type>, std::basic_string_view<char, std::char_traits<char>>&, String&&, String&&) @ 0x0000000014c52c2b
2961814:2025.10.21 00:19:12.987423 [ 1644 ] {} <Fatal> BaseDaemon: 12.0. inlined from ./ci/tmp/build/./src/Core/Block.cpp:43: void DB::onError<void, std::basic_string_view<char, std::char_traits<char>>&, String, String>(int, FormatStringHelperImpl<std::type_identity<std::basic_string_view<char, std::char_traits<char>>&>::type, std::type_iden
tity<String>::type, std::type_identity<String>::type>, std::basic_string_view<char, std::char_traits<char>>&, String&&, String&&)
2961817:2025.10.21 00:19:13.034426 [ 1644 ] {} <Fatal> BaseDaemon: 12. ./ci/tmp/build/./src/Core/Block.cpp:59: void DB::checkColumnStructure<void>(DB::ColumnWithTypeAndName const&, DB::ColumnWithTypeAndName const&, std::basic_string_view<char, std::char_traits<char>>, bool, int) @ 0x0000000014c46a76
2961937:2025.10.21 00:19:13.058626 [ 1644 ] {} <Fatal> BaseDaemon: 13. ./ci/tmp/build/./src/Core/Block.cpp:148: void DB::checkBlockStructure<void>(DB::Block const&, DB::Block const&, std::basic_string_view<char, std::char_traits<char>>, bool) @ 0x0000000014c4da73
2962001:2025.10.21 00:19:13.121107 [ 1644 ] {} <Fatal> BaseDaemon: 14. ./ci/tmp/build/./src/Processors/QueryPlan/AggregatingStep.cpp:582: DB::AggregatingStep::requestOnlyMergeForAggregateProjection(std::shared_ptr<DB::Block const> const&) @ 0x0000000019e76690
2962164:2025.10.21 00:19:13.225026 [ 1644 ] {} <Fatal> BaseDaemon: 15. ./ci/tmp/build/./src/Processors/QueryPlan/Optimizations/optimizeUseAggregateProjection.cpp:872: DB::QueryPlanOptimizations::optimizeUseAggregateProjections(DB::QueryPlan::Node&, std::list<DB::QueryPlan::Node, std::allocator<DB::QueryPlan::Node>>&, bool, bool, unsigned lon
g) @ 0x000000001a063422
2962220:2025.10.21 00:19:13.237292 [ 1644 ] {} <Fatal> BaseDaemon: 16. ./ci/tmp/build/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:265: DB::QueryPlanOptimizations::optimizeTreeSecondPass(DB::QueryPlanOptimizationSettings const&, DB::QueryPlan::Node&, std::list<DB::QueryPlan::Node, std::allocator<DB::QueryPlan::Node>>&, DB::Query
Plan&) @ 0x0000000019ff83f9
2962319:2025.10.21 00:19:13.284336 [ 1644 ] {} <Fatal> BaseDaemon: 17. ./ci/tmp/build/./src/Processors/QueryPlan/QueryPlan.cpp:506: DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) @ 0x0000000019f0d20a
2962346:2025.10.21 00:19:13.324338 [ 1644 ] {} <Fatal> BaseDaemon: 18. ./ci/tmp/build/./src/Processors/QueryPlan/QueryPlan.cpp:180: DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&, bool) @ 0x0000000019f0c742
2962360:2025.10.21 00:19:13.361363 [ 1644 ] {} <Fatal> BaseDaemon: 19. ./ci/tmp/build/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:395: DB::InterpreterSelectWithUnionQuery::execute() @ 0x00000000161199df
2962388:2025.10.21 00:19:13.402811 [ 1644 ] {} <Fatal> BaseDaemon: 20. ./ci/tmp/build/./src/Interpreters/executeQuery.cpp:1603: DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, std::unique_ptr<DB::ReadBuffer, std::default_delete<DB::ReadBuffer>>&, std::shared_ptr<DB:
:IAST>&, std::shared_ptr<DB::ImplicitTransactionControlExecutor>, std::function<void ()>) @ 0x00000000164679c7
2962446:2025.10.21 00:19:13.454081 [ 1644 ] {} <Fatal> BaseDaemon: 21. ./ci/tmp/build/./src/Interpreters/executeQuery.cpp:1812: DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000016462f92
2962483:2025.10.21 00:19:13.514300 [ 1644 ] {} <Fatal> BaseDaemon: 22. ./ci/tmp/build/./src/Server/TCPHandler.cpp:744: DB::TCPHandler::runImpl() @ 0x00000000199798ca
2962530:2025.10.21 00:19:13.606365 [ 1644 ] {} <Fatal> BaseDaemon: 23. ./ci/tmp/build/./src/Server/TCPHandler.cpp:2818: DB::TCPHandler::run() @ 0x0000000019996dd6
2962544:2025.10.21 00:19:13.609359 [ 1644 ] {} <Fatal> BaseDaemon: 24. ./ci/tmp/build/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x000000001e984247
2962553:2025.10.21 00:19:13.634915 [ 1644 ] {} <Fatal> BaseDaemon: 25. ./ci/tmp/build/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x000000001e9847fe
2962558:2025.10.21 00:19:13.647509 [ 1644 ] {} <Fatal> BaseDaemon: 26. ./ci/tmp/build/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x000000001e9239bf
2962601:2025.10.21 00:19:13.662447 [ 1644 ] {} <Fatal> BaseDaemon: 27. ./base/poco/Foundation/src/Thread_POSIX.cpp:341: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001e920fcf
2962603:2025.10.21 00:19:13.671584 [ 1644 ] {} <Fatal> BaseDaemon: 28. ? @ 0x0000000000094ac3
2962605:2025.10.21 00:19:13.671612 [ 1644 ] {} <Fatal> BaseDaemon: 29. ? @ 0x0000000000126850
2962923:2025.10.21 00:19:13.883622 [ 1644 ] {} <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: 91368276F46269AA566513A45B2D3477)
2965714:2025.10.21 00:19:16.157791 [ 1644 ] {} <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
2965719:2025.10.21 00:19:16.159151 [ 1644 ] {} <Fatal> BaseDaemon: Changed settings: min_compress_block_size = 2583694, max_compress_block_size = 1552085, max_block_size = 56176, min_external_table_block_size_bytes = 1, max_joined_block_size_rows = 52947, joined_block_split_single_row = true, max_insert_threads = 1, max_threads = 3, max_read
_buffer_size = 659042, connect_timeout_with_failover_ms = 2000, connect_timeout_with_failover_secure_ms = 3000, idle_connection_timeout = 36000, s3_max_get_rps = 1000000, s3_max_get_burst = 2000000, s3_max_put_rps = 1000000, s3_max_put_burst = 2000000, s3_check_objects_after_upload = true, max_remote_read_network_bandwidth = 1000000000000, m
ax_remote_write_network_bandwidth = 1000000000000, max_local_read_bandwidth = 1000000000000, max_local_write_bandwidth = 1000000000000, stream_like_engine_allow_direct_select = true, replication_wait_for_inactive_replica_timeout = 30, compile_expressions = false, min_count_to_compile_expression = 0, compile_sort_description = false, group_by
_two_level_threshold = 563021, group_by_two_level_threshold_bytes = 40443084, distributed_aggregation_memory_efficient = false, enable_memory_bound_merging_of_aggregation_results = false, allow_nonconst_timezone_arguments = true, min_chunk_bytes_for_parallel_parsing = 8416901, output_format_parallel_formatting = false, merge_tree_coarse_inde
x_granularity = 26, min_bytes_to_use_direct_io = 10737418240, min_bytes_to_use_mmap_io = 10737418240, use_skip_indexes_if_final = false, use_skip_indexes_if_final_exact_mode = false, use_skip_indexes_on_data_read = false, secondary_indices_enable_bulk_filtering = false, log_queries = true, insert_quorum_timeout = 60000, merge_tree_read_split
_ranges_into_intersecting_and_non_intersecting_injection_probability = 0.1899999976158142, http_response_buffer_size = 6272850, fsync_metadata = true, join_output_by_rowlist_perkey_rows_threshold = 0, distributed_ddl_task_timeout = 120, http_send_timeout = 60., http_receive_timeout = 60., use_index_for_in_with_subqueries_max_values = 1000000
000, opentelemetry_start_trace_probability = 0.10000000149011612, max_rows_to_read = 20000000, max_bytes_to_read = 1000000000000, max_bytes_to_read_leaf = 1000000000000, max_rows_to_group_by = 10000000000, max_bytes_ratio_before_external_group_by = 0.2, max_rows_to_sort = 10000000000, max_bytes_to_sort = 10000000000, prefer_external_sort_blo
ck_bytes = 100000000, max_bytes_ratio_before_external_sort = 0., max_bytes_before_remerge_sort = 1356766623, max_result_rows = 1000000000, max_result_bytes = 1000000000, max_execution_speed = 100000000000, max_execution_speed_bytes = 10000000000000, timeout_before_checking_execution_speed = 300., max_estimated_execution_time = 600., max_colu
mns_to_read = 20000, max_temporary_columns = 20000, max_temporary_non_const_columns = 20000, max_rows_in_set = 10000000000, max_bytes_in_set = 10000000000, max_rows_in_join = 10000000000, max_bytes_in_join = 10000000000, cross_join_min_rows_to_compress = 1, cross_join_min_bytes_to_compress = 100000000, max_rows_to_transfer = 1000000000, max_
bytes_to_transfer = 1000000000, max_rows_in_distinct = 10000000000, max_bytes_in_distinct = 10000000000, max_memory_usage = 5000000000, max_memory_usage_for_user = 32000000000, max_untracked_memory = 1048576, memory_profiler_step = 1048576, max_network_bandwidth = 100000000000, max_network_bytes = 1000000000000, max_network_bandwidth_for_use
r = 100000000000, max_network_bandwidth_for_all_users = 100000000000, max_temporary_data_on_disk_size_for_user = 100000000000, max_temporary_data_on_disk_size_for_query = 100000000000, max_backup_bandwidth = 100000000000, log_comment = 'f03571_block_structure_mismatch_after_metadata_only_alter.sql-test_9kxny9f9', send_logs_level = 'warning',
 optimize_read_in_order = false, optimize_aggregation_in_order = true, aggregation_in_order_max_block_bytes = 33100701, read_in_order_two_level_merge_threshold = 39, max_hyperscan_regexp_length = 1000000, max_hyperscan_regexp_total_length = 10000000, allow_introspection_functions = true, database_atomic_wait_for_drop_and_detach_synchronously
 = true, optimize_if_chain_to_multiif = true, lock_acquire_timeout = 60., query_cache_max_size_in_bytes = 10000000, query_cache_max_entries = 100000, use_query_condition_cache = false, database_replicated_initial_query_timeout_sec = 120, database_replicated_enforce_synchronous_settings = true, database_replicated_always_detach_permanently =
true, database_replicated_allow_replicated_engine_arguments = 3, distributed_ddl_output_mode = 'none', distributed_ddl_entry_format_version = 6, external_storage_max_read_rows = 10000000000, external_storage_max_read_bytes = 10000000000, local_filesystem_read_method = 'read', local_filesystem_read_prefetch = true, merge_tree_min_bytes_per_ta
sk_for_remote_reading = 1048576, merge_tree_compact_parts_min_granules_to_multibuffer_read = 19, async_insert_busy_timeout_max_ms = 5000, enable_filesystem_cache = true, enable_filesystem_cache_on_write_operations = true, throw_on_error_from_cache_on_write_operations = true, filesystem_cache_segments_batch_size = 50, page_cache_inject_evicti
on = true, load_marks_asynchronously = true, allow_prefetched_read_pool_for_remote_filesystem = false, allow_prefetched_read_pool_for_local_filesystem = false, filesystem_prefetch_step_marks = 50, filesystem_prefetch_max_memory_usage = 33554432, filesystem_prefetches_limit = 0, max_streams_for_merge_tree_reading = 1000, optimize_sorting_by_i
nput_stream_properties = false, insert_keeper_max_retries = 100, insert_keeper_retry_initial_backoff_ms = 1, insert_keeper_retry_max_backoff_ms = 10, insert_keeper_fault_injection_probability = 0.009999999776482582, optimize_distinct_in_order = false, parallel_replicas_local_plan = false, allow_experimental_analyzer = false, session_timezone
 = 'America/Mazatlan', optimize_extract_common_expressions = false

@devcrafter
Copy link
Member

@devcrafter devcrafter changed the title Revert "Revert "Add a test for #56334"" (but it shows a problem with parallel replicas) Revert "Revert "Add a test for #56334"" Dec 15, 2025
devcrafter and others added 5 commits December 15, 2025 12:03
…r-metadata-only-alter' of github.com:ClickHouse/ClickHouse into revert-84024-revert-84016-block-structure-mismatch-after-metadata-only-alter
…projection optimization

When using aggregate projection optimization (e.g., minmax_count projection),
the projection data may have different types for key columns due to metadata-only
ALTERs (e.g., extending an Enum). This caused a "Block structure mismatch" error
in `AggregatingStep::requestOnlyMergeForAggregateProjection`.

The fix adapts the projection input header to use the expected output types
for columns that exist in the output header. This ensures that after
`updateOutputHeader()` is called, the output header types match the expected
types from the original query.

Closes #56334

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@alexey-milovidov alexey-milovidov self-assigned this Jan 26, 2026
@alexey-milovidov alexey-milovidov added this pull request to the merge queue Jan 26, 2026
@alexey-milovidov
Copy link
Member Author

@amosbird, I'm a little bit doubtful about this. Maybe you can take a look.
The fix is 100% green, and the test now passes.

Merged via the queue into master with commit 7c3099f Jan 26, 2026
134 checks passed
@alexey-milovidov alexey-milovidov deleted the revert-84024-revert-84016-block-structure-mismatch-after-metadata-only-alter branch January 26, 2026 19:11
@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jan 26, 2026
@amosbird
Copy link
Collaborator

I'm a little bit doubtful about this. Maybe you can take a look.

I don’t see how this relates to projections in the tests, and I cannot reproduce any failures using fiddle across all major versions since 25.8.

@alexey-milovidov
Copy link
Member Author

@amosbird the test is intended to run on Debug, ASan, MSan, TSan, UBSan builds.

@alexey-milovidov
Copy link
Member Author

relates to projections in the tests

It's likely about "virtual projections".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants