Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase load_metadata_threads to 16 #57732

Merged
merged 2 commits into from Jan 2, 2024
Merged

Increase load_metadata_threads to 16 #57732

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
27f1963
Increase `load_metadata_threads` to 16
alexey-milovidov Dec 10, 2023
f0fb615
Update changed .reference
kssenii Dec 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/en/operations/storing-data.md
View file
Open in desktop
Expand Up @@ -196,7 +196,7 @@ These settings should be defined in the disk configuration section.

- `max_elements` - a limit for a number of cache files. Default: `10000000`.

- `load_metadata_threads` - number of threads being used to load cache metadata on starting time. Default: `1`.
- `load_metadata_threads` - number of threads being used to load cache metadata on starting time. Default: `16`.

File Cache **query/profile settings**:

Expand Down
2 changes: 1 addition & 1 deletion src/Interpreters/Cache/FileCache_fwd.h
View file
Open in desktop
Expand Up @@ -8,7 +8,7 @@ static constexpr int FILECACHE_DEFAULT_MAX_FILE_SEGMENT_SIZE = 32 * 1024 * 1024;
static constexpr int FILECACHE_DEFAULT_FILE_SEGMENT_ALIGNMENT = 4 * 1024 * 1024; /// 4Mi
static constexpr int FILECACHE_DEFAULT_BACKGROUND_DOWNLOAD_THREADS = 5;
static constexpr int FILECACHE_DEFAULT_BACKGROUND_DOWNLOAD_QUEUE_SIZE_LIMIT = 5000;
static constexpr int FILECACHE_DEFAULT_LOAD_METADATA_THREADS = 1;
static constexpr int FILECACHE_DEFAULT_LOAD_METADATA_THREADS = 16;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if cache directory is really small, it will slow down the load. (I checked). Therefore I left it as 1 by default. Do not remember exact numbers now, but probably we can neglect it...

static constexpr int FILECACHE_DEFAULT_MAX_ELEMENTS = 10000000;
static constexpr int FILECACHE_DEFAULT_HITS_THRESHOLD = 0;
static constexpr size_t FILECACHE_BYPASS_THRESHOLD = 256 * 1024 * 1024;
Expand Down
2 changes: 1 addition & 1 deletion tests/queries/0_stateless/02344_describe_cache.reference
View file
Open in desktop
@@ -1,2 +1,2 @@
1
102400 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/02344_describe_cache_test 5 5000 0 1
102400 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/02344_describe_cache_test 5 5000 0 16
4 changes: 2 additions & 2 deletions tests/queries/0_stateless/02908_filesystem_cache_as_collection.reference
View file
Open in desktop
@@ -1,2 +1,2 @@
1048576 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection_sql 5 5000 0 1
1048576 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection 5 5000 0 1
1048576 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection_sql 5 5000 0 16
1048576 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection 5 5000 0 16