Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: Rename Compaction to CompactionV2 #33858

Merged
merged 1 commit into from
Jun 16, 2024

Conversation

bigsheeper
Copy link
Contributor

Due to the removal of injection and syncSegments from the compaction, we need to ensure that no compaction is successfully executed during the rolling upgrade. This PR renames Compaction to CompactionV2, with the following effects:

  • New datacoord + old datanode: Utilizes the CompactionV2 interface, resulting in the datanode error "CompactionV2 not implemented," causing compaction to fail;
  • Old datacoord + new datanode: Utilizes the CompactionV1 interface, resulting in the datanode error "CompactionV1 not implemented," causing compaction to fail.

issue: #32809

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
@sre-ci-robot sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Jun 14, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Jun 14, 2024
@bigsheeper
Copy link
Contributor Author

test result:

  • New datacoord + old datanode:
    [2024/06/14 11:35:05.745 +08:00] [WARN] [retry/retry.go:106] ["retry func failed"] [retried=0] [error="service unimplemented: rpc error: code = Unimplemented desc = unknown method CompactionV2 for service milvus.proto.data.DataNode"]
  • Old datacoord + new datanode:
    [2024/06/14 11:32:15.741 +08:00] [WARN] [retry/retry.go:106] ["retry func failed"] [retried=0] [error="service unimplemented: rpc error: code = Unimplemented desc = unknown method Compaction for service milvus.proto.data.DataNode"]

Copy link

codecov bot commented Jun 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.91%. Comparing base (b69e909) to head (948c560).
Report is 10 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #33858      +/-   ##
==========================================
- Coverage   80.92%   80.91%   -0.01%     
==========================================
  Files        1058     1058              
  Lines      135395   135395              
==========================================
- Hits       109562   109556       -6     
+ Misses      21656    21648       -8     
- Partials     4177     4191      +14     
Files Coverage Δ
internal/datacoord/session_manager.go 69.91% <100.00%> (ø)
internal/datanode/services.go 81.99% <100.00%> (ø)
internal/distributed/datanode/client/client.go 92.75% <100.00%> (ø)
internal/distributed/datanode/service.go 82.29% <100.00%> (ø)
internal/util/mock/grpc_datanode_client.go 60.00% <100.00%> (ø)

... and 29 files with indirect coverage changes

@czs007
Copy link
Contributor

czs007 commented Jun 16, 2024

/approve
/lgtm

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bigsheeper, czs007

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit 8537f3d into milvus-io:master Jun 16, 2024
15 checks passed
bigsheeper added a commit to bigsheeper/milvus that referenced this pull request Jun 25, 2024
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
sre-ci-robot pushed a commit that referenced this pull request Jun 25, 2024
This PR cherry-picks the following commits:

- Implement task limit control logic in datanode.
#32881
- Load bf from storage instead of memory during L0 compaction.
#32913
- Remove dependencies on shards (e.g. SyncSegments, injection).
#33138
- Rename Compaction interface to CompactionV2.
#33858
- Remove the unused residual compaction logic.
#33932

issue: #32809

pr: #32881,
#32913,
#33138,
#33858,
#33932

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
yellow-shine pushed a commit to yellow-shine/milvus that referenced this pull request Jul 2, 2024
Due to the removal of injection and syncSegments from the compaction, we
need to ensure that no compaction is successfully executed during the
rolling upgrade. This PR renames Compaction to CompactionV2, with the
following effects:
- New datacoord + old datanode: Utilizes the CompactionV2 interface,
resulting in the datanode error "CompactionV2 not implemented," causing
compaction to fail;
- Old datacoord + new datanode: Utilizes the CompactionV1 interface,
resulting in the datanode error "CompactionV1 not implemented," causing
compaction to fail.

issue: milvus-io#32809

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/L Denotes a PR that changes 100-499 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants