[Feature Request]: Unify the background computation task running into task executor. #1182
Open
1 task done
Assignees
Labels
feature request
New feature or request
Comments
JinHai-CN
pushed a commit
that referenced
this issue
May 16, 2024
### What problem does this PR solve? 1. Refactor: implement compaction operator. 2. Refactor: use scheduler to execute compaction. 3. Refactor: fragment plan is a DAG now by connecting multiple plan tree. 4. Fix: Add delete to compaction todelete list in `SegmentEntry::Commit` instead of `SegmentEntry::DeleteData`, because it should be atomic with `SegmentEntry::max_row_ts_` update. Issue link:#1182 ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] Refactoring - [x] Test cases
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is there an existing issue for the same feature request?
Is your feature request related to a problem?
No response
Describe the feature you'd like
Currently some background tasks such as index building are not run in task executor, but a separate thread pool is started to run them. This is not good for unified management of compute resources. The goal of this feature request is to unify some background tasks and run them on the task executor. At the same time, the foreground tasks will have a higher priority, and can be scheduled and executed by the task executor first.
Describe implementation you've considered
No response
Documentation, adoption, use case
No response
Additional information
No response
The text was updated successfully, but these errors were encountered: