Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: limit the max pool size to 16 #30371

Merged
merged 1 commit into from
Jan 31, 2024

Conversation

yah01
Copy link
Member

@yah01 yah01 commented Jan 30, 2024

according to our benchmark, concurrency level 16 is enough to fully utilize the object storage network bandwidth

@sre-ci-robot sre-ci-robot added the size/S Denotes a PR that changes 10-29 lines. label Jan 30, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Jan 30, 2024
Copy link
Contributor

mergify bot commented Jan 30, 2024

@yah01 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

according to our benchmark, concurrency level 16 is enough to fully utilize the object storage network bandwidth

Signed-off-by: yah01 <yang.cen@zilliz.com>
Copy link

codecov bot commented Jan 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (fb9d64a) 79.83% compared to head (c7e30c2) 83.25%.
Report is 2 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #30371      +/-   ##
==========================================
+ Coverage   79.83%   83.25%   +3.42%     
==========================================
  Files         954      707     -247     
  Lines      134860   114892   -19968     
==========================================
- Hits       107660    95649   -12011     
+ Misses      23538    15579    -7959     
- Partials     3662     3664       +2     
Files Coverage Δ
internal/querynodev2/segments/pool.go 100.00% <100.00%> (ø)

... and 266 files with indirect coverage changes

@yanliang567 yanliang567 added ci-passed manual-pass manually set pass before ci-passed labeled labels Jan 31, 2024
Copy link
Contributor

@congqixia congqixia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia, yah01

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit 878c4c9 into milvus-io:master Jan 31, 2024
14 checks passed
yah01 added a commit to yah01/milvus that referenced this pull request Jan 31, 2024
according to our benchmark, concurrency level 16 is enough to fully
utilize the object storage network bandwidth

Signed-off-by: yah01 <yang.cen@zilliz.com>
sre-ci-robot pushed a commit that referenced this pull request Feb 20, 2024
according to our benchmark, concurrency level 16 is enough to fully
utilize the object storage network bandwidth
pr: #30371

Signed-off-by: yah01 <yang.cen@zilliz.com>
czs007 added a commit to czs007/milvus that referenced this pull request Feb 23, 2024
Revert "enhance: reduce many I/O operations while loading disk index (milvus-io#30189) (milvus-io#30690)"
This reverts commit d4c4bf9.

Revert "enhance: limit the max pool size to 16 (milvus-io#30371) (milvus-io#30415)"
This reverts commit 52ac071.

Revert "enhance: convert the `GetObject` util to async (milvus-io#30166) (milvus-io#30197)"
This reverts commit 4b7c5ba.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
yanliang567 pushed a commit that referenced this pull request Feb 24, 2024
Revert "enhance: reduce many I/O operations while loading disk index
(#30189) (#30690)" This reverts commit
d4c4bf9.

Revert "enhance: limit the max pool size to 16 (#30371) (#30415)" This
reverts commit 52ac071.

Revert "enhance: convert the `GetObject` util to async (#30166)
(#30197)" This reverts commit 4b7c5ba.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
MrPresent-Han added a commit to MrPresent-Han/milvus that referenced this pull request Mar 19, 2024
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
wayblink pushed a commit to wayblink/milvus that referenced this pull request Mar 20, 2024
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
MrPresent-Han added a commit to MrPresent-Han/milvus that referenced this pull request Mar 21, 2024
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

add vchannel to stats path
MrPresent-Han added a commit to MrPresent-Han/milvus that referenced this pull request Mar 21, 2024
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

add vchannel to stats path
MrPresent-Han added a commit to MrPresent-Han/milvus that referenced this pull request Mar 21, 2024
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

add vchannel to stats path
MrPresent-Han added a commit to MrPresent-Han/milvus that referenced this pull request Mar 21, 2024
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

add vchannel to stats path
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm manual-pass manually set pass before ci-passed labeled size/S Denotes a PR that changes 10-29 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants