Skip to content

Commit

Permalink
Improve search split cache docs (#4951)
Browse files Browse the repository at this point in the history
* Improve search split cache docs

* Address review comments
  • Loading branch information
rdettai committed May 22, 2024
1 parent 0db4cc5 commit ec502f0
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 6 deletions.
6 changes: 3 additions & 3 deletions docs/configuration/node-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,16 +208,16 @@ This section contains the configuration options for a Searcher.
| `partial_request_cache_capacity` | Partial request in memory cache capacity on a Searcher. Cache intermediate state for a request, possibly making subsequent requests faster. It can be disabled by setting the size to `0`. | `64M` |
| `max_num_concurrent_split_searches` | Maximum number of concurrent split search requests running on a Searcher. | `100` |
| `max_num_concurrent_split_streams` | Maximum number of concurrent split stream requests running on a Searcher. | `100` |
| `split_cache` | Searcher split cache configuration options defined in the section below. | |
| `split_cache` | Searcher split cache configuration options defined in the section below. Cache disabled if unspecified. | |


### Searcher split cache configuration

This section contains the configuration options for the searcher split cache.
This section contains the configuration options for the on disk searcher split cache.

| Property | Description | Default value |
| --- | --- | --- |
| `max_num_bytes` | Maximum size in bytes allowed in the split cache. | `1G` |
| `max_num_bytes` | Maximum disk size in bytes allowed in the split cache. Can be exceeded by the size of one split. | |
| `max_num_splits` | Maximum number of splits allowed in the split cache. | `10000` |
| `num_concurrent_downloads` | Maximum number of concurrent download of splits. | `1` |

Expand Down
7 changes: 4 additions & 3 deletions quickwit/quickwit-storage/src/split_cache/split_table.rs
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,8 @@ pub struct SplitInfo {
/// - downloading_splits
/// - candidate_splits.
///
/// It is possible for the split table to exceed its limits, by at most one split.
/// It is possible for the split table size in bytes to exceed its limits, by at
/// most one split.
pub struct SplitTable {
on_disk_splits: BTreeSet<SplitKey>,
downloading_splits: BTreeSet<SplitKey>,
Expand Down Expand Up @@ -360,7 +361,7 @@ impl SplitTable {
return false;
}
if self.on_disk_splits.len() + self.downloading_splits.len()
> self.limits.max_num_splits.get() as usize
>= self.limits.max_num_splits.get() as usize
{
return true;
}
Expand Down Expand Up @@ -608,7 +609,7 @@ mod tests {
splits_to_delete,
split_to_download,
} = split_table.find_download_opportunity().unwrap();
assert_eq!(&splits_to_delete[..], &[splits[0].0][..]);
assert_eq!(&splits_to_delete[..], &[splits[0].0, splits[1].0]);
assert_eq!(split_to_download.split_ulid, new_ulid);
}

Expand Down

0 comments on commit ec502f0

Please sign in to comment.