Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compaction rate limit #12312

Merged
merged 13 commits into from
Apr 19, 2024
Merged

Conversation

shuwenwei
Copy link
Contributor

Description

  1. add config items 'compaction_read_throughput_mb_per_sec' and 'compaction_read_operation_per_sec'.
  2. CompactionScheduler and RepairData is constrained by these config items
  3. reimplement compaction write rate limit.

Copy link
Contributor

@jt2594838 jt2594838 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is generally good, but with some minor issues that would best be fixed.

@OneSizeFitsQuorum OneSizeFitsQuorum merged commit b9e9aaa into apache:master Apr 19, 2024
55 of 57 checks passed
shuwenwei added a commit to shuwenwei/iotdb that referenced this pull request May 21, 2024
* add compaction read rate limiter

* modify method name

* modify default compaction read rate limit

* add config item to limit compaction read speed

* add config item to limit compaction read speed

* fix throughput rate is not correct

* fix some issues

* compaction read rate has no limit in default config

* use correct unit when init rate limiter

* modify maxSizePerWrite

* remove useless code
HTHou pushed a commit that referenced this pull request May 21, 2024
* Fix uncompress byte buffer in RepairDataFileScanUtil

* Modify compaction task priority comparator (#12255)

* delay estimate memory of InnerSpaceCompactionTask (#12314)

* Compaction rate limit (#12312)

* add compaction read rate limiter

* modify method name

* modify default compaction read rate limit

* add config item to limit compaction read speed

* add config item to limit compaction read speed

* fix throughput rate is not correct

* fix some issues

* compaction read rate has no limit in default config

* use correct unit when init rate limiter

* modify maxSizePerWrite

* remove useless code
HTHou pushed a commit that referenced this pull request Jun 26, 2024
* add compaction read rate limiter

* modify method name

* modify default compaction read rate limit

* add config item to limit compaction read speed

* add config item to limit compaction read speed

* fix throughput rate is not correct

* fix some issues

* compaction read rate has no limit in default config

* use correct unit when init rate limiter

* modify maxSizePerWrite

* remove useless code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants