Skip to content

fix: Fix remaining issues with Datanode pooling and StorageV2 #43147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jul 10, 2025
Merged

fix: Fix remaining issues with Datanode pooling and StorageV2 #43147

merged 3 commits into from
Jul 10, 2025

Conversation

xiaocai2333
Copy link
Contributor

issue: #43146

@sre-ci-robot sre-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines. label Jul 4, 2025
@sre-ci-robot sre-ci-robot requested review from cydrain and czs007 July 4, 2025 06:43
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Jul 4, 2025
@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from e507239 to 585c2f1 Compare July 4, 2025 06:48
bigsheeper
bigsheeper reviewed Jul 4, 2025
View reviewed changes
@codecov Codecov
Copy link

codecov bot commented Jul 4, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 76.92308% with 9 lines in your changes missing coverage. Please review.

Project coverage is 79.79%. Comparing base (c4634d8) to head (60ca615).
Report is 13 commits behind head on master.

Files with missing lines Patch % Lines
internal/datacoord/import_util.go 53.84% 4 Missing and 2 partials ⚠️
internal/flushcommon/syncmgr/pack_writer_v2.go 62.50% 2 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #43147      +/-   ##
==========================================
+ Coverage   78.92%   79.79%   +0.87%     
==========================================
  Files        1562     1217     -345     
  Lines      222864   190547   -32317     
==========================================
- Hits       175897   152051   -23846     
+ Misses      40572    32103    -8469     
+ Partials     6395     6393       -2
Components Coverage Δ
Client 79.39% <ø> (ø)
Core ∅ <ø> (∅)
Go 79.90% <76.92%> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
internal/datanode/importv2/task_import.go 70.49% <100.00%> (+0.16%) ⬆️
internal/datanode/importv2/task_l0_import.go 69.23% <100.00%> (+0.19%) ⬆️
internal/datanode/importv2/util.go 77.28% <100.00%> (+0.10%) ⬆️
internal/datanode/index/task_stats.go 55.82% <100.00%> (+0.09%) ⬆️
internal/flushcommon/syncmgr/options.go 94.44% <100.00%> (+0.32%) ⬆️
internal/flushcommon/syncmgr/task.go 92.56% <100.00%> (+0.06%) ⬆️
internal/flushcommon/syncmgr/pack_writer_v2.go 78.86% <62.50%> (-1.49%) ⬇️
internal/datacoord/import_util.go 74.66% <53.84%> (-0.49%) ⬇️

... and 360 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from 585c2f1 to 0017b7f Compare July 7, 2025 02:58
czs007
czs007 approved these changes Jul 7, 2025
View reviewed changes
Copy link
Collaborator

@czs007 czs007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 7, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

/run-cpu-e2e

1 similar comment
@xiaocai2333
Copy link
Contributor Author

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 7, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from 0017b7f to 90c877b Compare July 7, 2025 10:50
@sre-ci-robot sre-ci-robot removed the lgtm label Jul 7, 2025
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 7, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

1 similar comment
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 7, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from 40794b2 to 0bd23ec Compare July 8, 2025 03:01
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 8, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from 0bd23ec to d9321ef Compare July 8, 2025 12:35
@xiaocai2333 xiaocai2333 changed the title enhance: Support storage version for import fix: Fix remaining issues with Datanode pooling and StorageV2 Jul 8, 2025
@mergify mergify bot added the kind/bug Issues or changes related a bug label Jul 8, 2025
czs007
czs007 approved these changes Jul 8, 2025
View reviewed changes
Copy link
Collaborator

@czs007 czs007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: czs007, xiaocai2333

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from d9321ef to 9f97683 Compare July 8, 2025 12:50
@czs007
Copy link
Collaborator

czs007 commented Jul 8, 2025

/lgtm

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 8, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333
Fix remaining issues with Datanode pooling and StorageV2
cb3d9a4 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
@xiaocai2333
fix ut
2e76656 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
@xiaocai2333 xiaocai2333 force-pushed the add_storage_version_import branch from 9f97683 to 2e76656 Compare July 9, 2025 01:46
@sre-ci-robot sre-ci-robot removed the lgtm label Jul 9, 2025
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@xiaocai2333
fix it
60ca615 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
@czs007
Copy link
Collaborator

czs007 commented Jul 9, 2025

/lgtm

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

rerun cpp-unit-test

@xiaocai2333
Copy link
Contributor Author

rerun go-sdk

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

rerun cpp-unit-test

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 9, 2025

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

rerun cpp-unit-test

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 10, 2025

@xiaocai2333 cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@xiaocai2333
Copy link
Contributor Author

rerun cpp-unit-test

1 similar comment
@xiaocai2333
Copy link
Contributor Author

rerun cpp-unit-test

@czs007 czs007 added ci-passed manual-pass manually set pass before ci-passed labeled labels Jul 10, 2025
@sre-ci-robot sre-ci-robot merged commit 3ffd44f into milvus-io:master Jul 10, 2025
18 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bigsheeper bigsheeper bigsheeper left review comments

@czs007 czs007 czs007 approved these changes

@cydrain cydrain Awaiting requested review from cydrain

Assignees

@czs007 czs007

Labels
approved ci-passed dco-passed DCO check passed. kind/bug Issues or changes related a bug kind/enhancement Issues or changes related to enhancement lgtm manual-pass manually set pass before ci-passed labeled size/XXL Denotes a PR that changes 1000+ lines.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@xiaocai2333 @sre-ci-robot @czs007 @bigsheeper