Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: Enable channel exclusive balance policy #32911

Merged
merged 1 commit into from
May 10, 2024

Conversation

weiliu1031
Copy link
Contributor

issue: #32910

  • split replica's node list to channels when create replicas
  • balance nodes among channels when node change happens
  • implement channel level balance, let balance happens in channel level

@sre-ci-robot sre-ci-robot added area/internal-api area/test sig/testing test/integration integration test size/XXL Denotes a PR that changes 1000+ lines. labels May 9, 2024
@mergify mergify bot added the dco-passed DCO check passed. label May 9, 2024
Copy link
Contributor

mergify bot commented May 9, 2024

@weiliu1031

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly 

Please review and update your PR to comply with these guidelines.

 * split replica's node list to channels when create replicas
 * balance nodes among channels when node change happens
 * implement channel level balance, let balance happens in channel level

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
@weiliu1031 weiliu1031 changed the title Enable channel exclusive balance policy enhance: Enable channel exclusive balance policy May 9, 2024
@mergify mergify bot added kind/enhancement Issues or changes related to enhancement and removed do-not-merge/invalid-pr-format labels May 9, 2024
Copy link

codecov bot commented May 9, 2024

Codecov Report

Attention: Patch coverage is 89.01099% with 30 lines in your changes are missing coverage. Please review.

Project coverage is 81.84%. Comparing base (efa58ae) to head (0846829).
Report is 13 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #32911      +/-   ##
==========================================
+ Coverage   81.82%   81.84%   +0.02%     
==========================================
  Files        1006     1007       +1     
  Lines      126831   127071     +240     
==========================================
+ Hits       103778   104001     +223     
- Misses      19126    19127       +1     
- Partials     3927     3943      +16     
Files Coverage Δ
internal/querycoordv2/balance/balance.go 96.96% <ø> (ø)
internal/querycoordv2/checkers/channel_checker.go 85.21% <100.00%> (+0.65%) ⬆️
internal/querycoordv2/job/job_load.go 90.90% <100.00%> (+0.23%) ⬆️
internal/querycoordv2/meta/replica.go 100.00% <100.00%> (ø)
internal/querycoordv2/meta/replica_manager.go 82.01% <100.00%> (+0.56%) ⬆️
internal/querycoordv2/utils/meta.go 55.95% <100.00%> (-1.20%) ⬇️
pkg/util/paramtable/component_param.go 98.46% <100.00%> (+<0.01%) ⬆️
internal/querycoordv2/checkers/segment_checker.go 87.30% <60.00%> (-2.19%) ⬇️
internal/querycoordv2/server.go 82.35% <40.00%> (-1.19%) ⬇️
...erycoordv2/balance/channel_level_score_balancer.go 88.81% <88.81%> (ø)

... and 31 files with indirect coverage changes

@mergify mergify bot added the ci-passed label May 9, 2024
@yanliang567 yanliang567 added this to the 2.4.2 milestone May 10, 2024
@xiaofan-luan
Copy link
Contributor

/lgtm
/approve

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: weiliu1031, xiaofan-luan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit e2332bd into milvus-io:master May 10, 2024
15 checks passed
weiliu1031 added a commit to weiliu1031/milvus that referenced this pull request May 10, 2024
issue: milvus-io#32910  
* split replica's node list to channels when create replicas
 * balance nodes among channels when node change happens
 * implement channel level balance, let balance happens in channel level

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
UnyieldingOrca pushed a commit to UnyieldingOrca/milvus that referenced this pull request May 10, 2024
issue: milvus-io#32910  
* split replica's node list to channels when create replicas
 * balance nodes among channels when node change happens
 * implement channel level balance, let balance happens in channel level

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
sre-ci-robot pushed a commit that referenced this pull request May 14, 2024
issue: #32910
see also: #32911
when channel exclusive mode is enabled, replica will record channel node
info in meta store, and if the balance policy changes, which means
channel exclusive mode is disabled, we should clean up the channel node
info in meta store, and stop to balance node between channels.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved area/internal-api area/test ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm sig/testing size/XXL Denotes a PR that changes 1000+ lines. test/integration integration test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants