Skip to content

Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries#89640

Merged
azat merged 1 commit intoClickHouse:masterfrom
azat:unique-filesystem-caches
Nov 14, 2025
Merged

Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries#89640
azat merged 1 commit intoClickHouse:masterfrom
azat:unique-filesystem-caches

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Nov 6, 2025

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix wrong FilesystemCacheBytes (and others) in system.asynchronous_metrics. Run SYSTEM queries on filesystem caches only once. Atomic view for caches that points to the same path in system.filesystem_caches.

Details

It is possible to have filesystem cache i.e. "a" and "b", that points to the same path.

Previously those was accounted multiple times in FilesystemCacheBytes from system.asynchronous_metrics, which is wrong.

Also SYSTEM queries executed actions on the same caches multiple times in this case, which is redundant.

Plus system.filesystem_caches may report different set of files for various caches even though they share the same path, fix this as well.

@azat azat requested a review from kssenii November 6, 2025 12:34
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Nov 6, 2025

Workflow [PR], commit [01bacf6]

Summary:

job_name test_name status info comment
BuzzHouse (amd_debug) failure
Buzzing result failure cidb
BuzzHouse (arm_asan) failure
Buzzing result failure cidb

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Nov 6, 2025
@kssenii kssenii added the pr-must-backport Pull request should be backported intentionally. Use this label with great care! label Nov 14, 2025
@azat azat enabled auto-merge November 14, 2025 10:50
@azat azat disabled auto-merge November 14, 2025 10:51
… queries

It is possible to have filesystem cache i.e. "a" and "b", that points to
the same path.

Previously those was accounted multiple times in FilesystemCacheBytes
from system.asynchronous_metrics, which is wrong.

Also SYSTEM queries executed actions on the same caches multiple times
in this case, which is redundant.

Plus system.filesystem_caches may report different set of files for
various caches even though they share the same path, fix this as well.
@azat azat force-pushed the unique-filesystem-caches branch from e7fdae1 to 01bacf6 Compare November 14, 2025 10:53
@azat azat added this pull request to the merge queue Nov 14, 2025
Merged via the queue into ClickHouse:master with commit ea95cb1 Nov 14, 2025
127 of 130 checks passed
@azat azat deleted the unique-filesystem-caches branch November 14, 2025 15:42
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Nov 14, 2025
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label Nov 14, 2025
robot-ch-test-poll added a commit that referenced this pull request Nov 14, 2025
Cherry pick #89640 to 25.8: Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries
robot-clickhouse added a commit that referenced this pull request Nov 14, 2025
robot-ch-test-poll added a commit that referenced this pull request Nov 14, 2025
Cherry pick #89640 to 25.9: Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries
robot-clickhouse added a commit that referenced this pull request Nov 14, 2025
robot-ch-test-poll added a commit that referenced this pull request Nov 14, 2025
Cherry pick #89640 to 25.10: Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries
robot-clickhouse added a commit that referenced this pull request Nov 14, 2025
clickhouse-gh bot added a commit that referenced this pull request Nov 14, 2025
Backport #89640 to 25.10: Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries
azat added a commit that referenced this pull request Nov 16, 2025
Backport #89640 to 25.9: Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries
azat added a commit that referenced this pull request Nov 16, 2025
Backport #89640 to 25.8: Remove duplicated filesystem caches from asynchronous metrics, SYSTEM queries
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Nov 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-improvement Pull request with some product improvements pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants