enhance: [cherry-pick] Periodically synchronize segments to datanode watcher (#33420) #34186
Conversation
|
@xiaocai2333 Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco. |
mergify
bot
added
needs-dco
xiaocai2333
force-pushed
the
pick_sync_segment
branch
from
16b4011
to
b5c0ed4
Compare
mergify
bot
added
dco-passed
| Key: "dataCoord.sync.interval", | ||
| Version: "2.4.3", | ||
| Doc: "The time interval for regularly syncing segments", | ||
| DefaultValue: "600", // 10 * 60 seconds |
| newSegments := make([]*datapb.SyncSegmentInfo, 0, len(missingSegments)) | ||
| futures := make([]*conc.Future[any], 0, len(missingSegments)) | ||
|
|
||
| for _, segID := range missingSegments { |
There was a problem hiding this comment.
The timeout for the SyncSegments RPC is 10 seconds. If there are many segments, it may time out due to loading
many stats log.
There was a problem hiding this comment.
will fix it in next pr, master also need it.
xiaocai2333
force-pushed
the
pick_sync_segment
branch
from
b5c0ed4
to
90b6e72
Compare
…s-io#33420) issue: milvus-io#32809 --------- Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
xiaocai2333
force-pushed
the
pick_sync_segment
branch
from
90b6e72
to
1d88fa3
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 2.4 #34186 +/- ##
==========================================
+ Coverage 81.96% 81.99% +0.03%
==========================================
Files 1013 1013
Lines 130223 130382 +159
==========================================
+ Hits 106737 106907 +170
+ Misses 19494 19487 -7
+ Partials 3992 3988 -4
|
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: czs007, xiaocai2333 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This PR primary picks up the SyncSegments functionality, including the following commits:
issue: #32809
master pr: #33420, #33664, #33829, #34056, #34156