Skip to content

polish compaction filter configuration doc #16179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tonyxuqqi wants to merge 1 commit into from
Closed

polish compaction filter configuration doc #16179

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion garbage-collection-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,8 @@ show config where type = 'tikv' and name like '%enable-compaction-filter%';
| tikv | 172.16.5.36:20163 | gc.enable-compaction-filter | true |
| tikv | 172.16.5.35:20163 | gc.enable-compaction-filter | true |
+------+-------------------+-----------------------------+-------+
```
```

> **Note：**
>
> The compaction filter may delay the entries to be deleted in some cases, which could lead to performance regression in TiKV scan. And if `next` or `prev` in [`Coprocessor Details->Total Ops Details`](/grafana-tikv-dashboard.md###Coprocessor-Detail) are far more (3x+) than `processed_keys`, then it's recommended to disable compaction filter to speed up the GC. In TiDB 7.1.3 or newer version, TiDB would check each region's redundent version count[`region-compact-min-redundant-rows`](/tikv-configuration-file.md###region-compact-min-redundant-rows) and ratio[`region-compact-redundant-rows-percent`](/tikv-configuration-file.md###region-compact-redundant-rows-percent) to automatic trigger compaction, which would greatly mitigate the issue. And therefore in TiDB 7.1.3 or newer version, it's recommended to tune these parameters instead of disabling compaction filter.
Comment on lines +112 to +114
Copy link
Collaborator

@Oreoxmt Oreoxmt Jan 22, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
> **Note：**
>
> The compaction filter may delay the entries to be deleted in some cases, which could lead to performance regression in TiKV scan. And if `next` or `prev` in [`Coprocessor Details->Total Ops Details`](/grafana-tikv-dashboard.md###Coprocessor-Detail) are far more (3x+) than `processed_keys`, then it's recommended to disable compaction filter to speed up the GC. In TiDB 7.1.3 or newer version, TiDB would check each region's redundent version count[`region-compact-min-redundant-rows`](/tikv-configuration-file.md###region-compact-min-redundant-rows) and ratio[`region-compact-redundant-rows-percent`](/tikv-configuration-file.md###region-compact-redundant-rows-percent) to automatic trigger compaction, which would greatly mitigate the issue. And therefore in TiDB 7.1.3 or newer version, it's recommended to tune these parameters instead of disabling compaction filter.
> **Note:**
>
> When using the mechanism of GC in Compaction Filter, the GC progress might be delayed, which leads to the performance regression of TiKV scan. If your workload contains a large number of coprocessor requests, and you observe that the number of `next()` or `prev()` calls of Total Ops Details in the [**TiKV-Details > Coprocessor Detail**](/grafana-tikv-dashboard.md#coprocessor-detail) panel is three times more than the number of `processed_keys` calls, consider the following actions:
> - For TiDB v7.1.3 earlier versions, it is recommended to try disabling Compaction Filter to speed up GC.
> - Starting from v7.1.3, to speed up GC in Compaction Filter, TiDB automatically triggers compaction based on the redundant version number [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows) and ratio [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent) of each Region. For TiDB v7.1.3 and later versions, if you encounter the preceding situation, it is recommended to adjust these two parameters instead of disabling Compaction Filter.