Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Major compaction feature #32903

Closed
wants to merge 36 commits into from
Closed

feat: Major compaction feature #32903

wants to merge 36 commits into from

Conversation

wayblink
Copy link
Contributor

@wayblink wayblink commented May 9, 2024 •
edited
Loading

@sre-ci-robot sre-ci-robot added the do-not-merge/work-in-progress Don't merge even CI passed. label May 9, 2024
@sre-ci-robot sre-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines. label May 9, 2024
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: wayblink
To complete the pull request process, please assign czs007 after the PR has been reviewed.
You can assign the PR to them by writing /assign @czs007 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mergify mergify bot added the needs-dco DCO is missing in this pull request. label May 9, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly

Please review and update your PR to comply with these guidelines.

@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@wayblink wayblink force-pushed the merge-major branch 2 times, most recently from 5a3fab7 to 95f187f Compare May 9, 2024 10:20
@mergify mergify bot added dco-passed DCO check passed. and removed needs-dco DCO is missing in this pull request. labels May 9, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink ut workflow job failed, comment rerun ut can trigger the job again.

@mergify mergify bot added needs-dco DCO is missing in this pull request. and removed dco-passed DCO check passed. labels May 9, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink ut workflow job failed, comment rerun ut can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented May 9, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify mergify bot added dco-passed DCO check passed. and removed needs-dco DCO is missing in this pull request. labels May 9, 2024
wayblink and others added 18 commits June 3, 2024 15:49
@wayblink
refine datacoord compaction
4e55ea5
@wayblink
add metric
965e95c
@wayblink
fix potential bug
52c3ce8
@wayblink
add count
cd0239a
@wayblink
fix some ut
a38bc0a
@wayblink
simplify compaction state
3cd4bf1
@wayblink
Deal with clustering data skew and use proto to pass analyze info in cgo
43734dd 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
@chasingegg @wayblink
Fix autoindex optimization
a7ebc11 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
@wayblink
repair ut and code styles
22353f0
@wayblink
fix uts
8a1a013
@chasingegg @wayblink
Remove cgo params
fcce24a 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
@wayblink
fix ut
f661eda
@wayblink
split out compactionTaskMeta
6bd2b9b 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
@wayblink
remove syncSegments from compaction
742717a
@wayblink
refine code
8bc3a8b
@wayblink
Refine datacoord meta interface
5280452
@wayblink
setup clustering_compactor_test
8f7dece 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
@wayblink
clustering compaction integration test
85923ec 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 3, 2024

@wayblink ut workflow job failed, comment rerun ut can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 3, 2024

@wayblink ut workflow job failed, comment rerun ut can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 3, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@wayblink
fix ut
79680fd 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 3, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jun 3, 2024

@wayblink E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@czs007
Copy link
Collaborator

czs007 commented Jun 7, 2024

duplicated with #33620

@czs007 czs007 closed this Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xiaofan-luan xiaofan-luan xiaofan-luan left review comments

@czs007 czs007 czs007 left review comments

@bigsheeper bigsheeper bigsheeper left review comments

@congqixia congqixia Awaiting requested review from congqixia

Assignees
No one assigned
Labels
area/compilation area/dependency Pull requests that update a dependency file area/internal-api area/test kind/feature Issues related to feature request from users needs-dco DCO is missing in this pull request. sig/testing size/XXL Denotes a PR that changes 1000+ lines. test/integration integration test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@wayblink @sre-ci-robot @czs007 @bigsheeper @xiaofan-luan @xiaocai2333 @chasingegg