Skip to content

Fix infinite loop in AzureObjectStorage::listObjects#90947

Merged
kssenii merged 2 commits intoClickHouse:masterfrom
jkartseva:remove-redundant-list-blobs
Nov 27, 2025
Merged

Fix infinite loop in AzureObjectStorage::listObjects#90947
kssenii merged 2 commits intoClickHouse:masterfrom
jkartseva:remove-redundant-list-blobs

Conversation

@jkartseva
Copy link
Copy Markdown
Member

@jkartseva jkartseva commented Nov 27, 2025

The loop calls client_ptr->ListBlobs(options) inside the body on every iteration, overwriting blob_list_response with the first page.
This cancels out the pagination logic (MoveToNextPage()) in the loop. If the first page returns HasPage() == true, for example, when the first page is empty, this leads to an infinite loop:
https://pastila.clickhouse.com/?00079d20/bac6ffc01386edeba163db1798cf2832#zb4gt9R5wmh4zOzbk3PbeA==

Azurite does not reproduce this.
I’ve tried to reproduce the bug using gtest mocks (e.g., for the AzureBlobStorage::ContainerClient), but haven’t succeeded yet.

For reference, the SDK demonstrates blob iteration as follows:
https://github.com/ClickHouse/azure-sdk-for-cpp/blob/0f7a2013f7d79058047fc4bd35e94d20578c0d2b/sdk/storage/azure-storage-blobs/test/ut/blob_container_client_test.cpp#L299-L306

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix possible infinite loop in azure list blobs.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@jkartseva jkartseva added the can be tested Allows running workflows for external contributors label Nov 27, 2025
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Nov 27, 2025

Workflow [PR], commit [a4957d0]

Summary:

job_name test_name status info comment
Stateless tests (arm_asan, targeted) failure
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
00446_clear_column_in_partition_concurrent_zookeeper FAIL cidb
Stateless tests (amd_binary, old analyzer, s3 storage, DatabaseReplicated, parallel) failure
00075_shard_formatting_negate_of_negative_literal FAIL cidb
03321_inner_materialized_view_nested FAIL cidb
Fatal messages (in clickhouse-server.log or clickhouse-server.err.log) FAIL cidb
Stateless tests (amd_msan, parallel) failure
00596_limit_on_expanded_ast FAIL cidb, flaky
Integration tests (amd_binary, 3/5) failure
test_refreshable_mat_view_replicated/test.py::test_real_wait_refresh[to_clause0-False-False] FAIL cidb, flaky
Integration tests (amd_binary, 4/5) failure
test_refreshable_mv_skip_old_temp_table_ddls/test.py::test_refreshable_mv_skip_old_temp_tables_ddls[1-True-False] FAIL cidb, flaky
Integration tests (amd_binary, 5/5) failure
test_merge_tree_s3/test.py::test_merge_canceled_by_s3_errors[node-broken_s3] FAIL cidb, flaky
BuzzHouse (amd_debug) failure
Logical error: 'Inconsistent AST formatting: the query: FAIL cidb
BuzzHouse (amd_ubsan) failure
UndefinedBehaviorSanitizer: undefined behavior (STID: 1486-329a) FAIL cidb
Performance Comparison (amd_release, master_head, 2/6) failure
Start failure
Performance Comparison (amd_release, master_head, 3/6) failure
Start failure

@clickhouse-gh clickhouse-gh bot added the pr-not-for-changelog This PR should not be mentioned in the changelog label Nov 27, 2025
@kssenii kssenii self-assigned this Nov 27, 2025
if (client_ptr->IsClientForDisk())
ProfileEvents::increment(ProfileEvents::DiskAzureListObjects);

blob_list_response = client_ptr->ListBlobs(options);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest I do not understand how it worked before at all, it looks like because of this blob_list_response override inside the loop we always listed only the first page and never went to the next one?

@kssenii kssenii added this pull request to the merge queue Nov 27, 2025
@kssenii kssenii added pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-bugfix Pull request with bugfix, not backported by default and removed pr-not-for-changelog This PR should not be mentioned in the changelog labels Nov 27, 2025
Merged via the queue into ClickHouse:master with commit 5372318 Nov 27, 2025
119 of 130 checks passed
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Nov 27, 2025
@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label Nov 27, 2025
jkartseva added a commit that referenced this pull request Nov 28, 2025
Cherry pick #90947 to 25.8: Fix infinite loop in AzureObjectStorage::listObjects
jkartseva added a commit that referenced this pull request Nov 28, 2025
Cherry pick #90947 to 25.10: Fix infinite loop in AzureObjectStorage::listObjects
jkartseva added a commit that referenced this pull request Nov 28, 2025
Cherry pick #90947 to 25.11: Fix infinite loop in AzureObjectStorage::listObjects
robot-ch-test-poll1 added a commit that referenced this pull request Nov 28, 2025
Cherry pick #90947 to 25.9: Fix infinite loop in AzureObjectStorage::listObjects
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Nov 28, 2025
jkartseva added a commit that referenced this pull request Dec 1, 2025
Backport #90947 to 25.8: Fix infinite loop in AzureObjectStorage::listObjects
jkartseva added a commit that referenced this pull request Dec 1, 2025
Backport #90947 to 25.9: Fix infinite loop in AzureObjectStorage::listObjects
jkartseva added a commit that referenced this pull request Dec 1, 2025
Backport #90947 to 25.10: Fix infinite loop in AzureObjectStorage::listObjects
jkartseva added a commit that referenced this pull request Dec 1, 2025
Backport #90947 to 25.11: Fix infinite loop in AzureObjectStorage::listObjects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-bugfix Pull request with bugfix, not backported by default pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants