replace binlogIO with io.BinlogIO in datanode #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wayblink
force-pushed
the
binlogio
branch
5 times, most recently
from
f1977d9
to
b868dc1
Compare
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
pr: milvus-io#29675 Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
…o#29858) See also milvus-io#29803 This PR: - Add trace span for `LoadIndex` & `LoadFieldData` in segment loader - Add `TraceCtx` parameter for `Index.Load` in segcore - Add span for ReadFiles & Engine Load for Memory/Disk Vector index --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
this supports mmap for marisa trie index related milvus-io#21866 Signed-off-by: yah01 <yang.cen@zilliz.com>
…9860) milvus-io#29759 Signed-off-by: luzhang <luzhang@zilliz.com> Co-authored-by: luzhang <luzhang@zilliz.com>
issue: milvus-io#29793 Use `DocSetCollector` instead of `TopDocsCollector`, which will avoid scoring and sorting. --------- Signed-off-by: longjiquan <jiquan.long@zilliz.com>
this protect the loading index from failure, and speed up the loading progress Signed-off-by: yah01 <yang.cen@zilliz.com>
If segment has more than 128 log fils, drop segment will exceed etcd txn ops limit, which will failed the drop segment request This PR drop segment meta info with prefix, to avoid drop segment meta failed --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: milvus-io#29709 #291712 to avoid concurrent recursive RLock and Lock cause deadlock, This PR remove the unnecessary lock in config manager --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>
…9873) This pull request simplifies the integration test for cross-cluster routing by reusing `integration.MiniClusterSuite`, instead of defining custom Milvus clients, servers, and etcd client. issue: milvus-io#29874 Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
…#29870) this leads to the EntitiesNum metric would be never reduced fix: milvus-io#29766 Signed-off-by: yah01 <yang.cen@zilliz.com>
issue：milvus-io#22837 Add bfloat16 vector, add the index part of float16 vector. Signed-off-by: Writer-X <1256866856@qq.com>
milvus-io#29759 Signed-off-by: luzhang <luzhang@zilliz.com> Co-authored-by: luzhang <luzhang@zilliz.com>
This PR change some frequency log to rated level Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
…9641) milvus-io#22882 /kind feature Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: nico <cheng.yuan@zilliz.com>
pr: milvus-io#29634 After verifying that the problem is fixed, it is necessary to run this case every night for regression. Signed-off-by: elstic <hao.wang@zilliz.com>
Update Pytest image changes See changes: milvus-io@930735c Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
issue: milvus-io#29846 Signed-off-by: jaime <yun.zhang@zilliz.com>
…e up (milvus-io#29890) milvus-io#29889 Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Update Knowhere Commit Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Add tracing span for search/query operation waiting tsafe duration Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
milvus-io#29912) related milvus-io#29843 Signed-off-by: yah01 <yang.cen@zilliz.com>
…o#30500) See also milvus-io#27675 milvus-io#30469 For a sync task, the segment could be compacted during sync task. In previous implementation, this sync task will hold only the old segment id as KeyLock, in which case compaction on compacted to segment may run in parallel with delta sync of this sync task. This PR introduces sync target segment verification logic. It shall check target segment lock it's holding beforing actually syncing logic. If this check failed, sync task shall return`errTargetSegementNotMatch` error and make manager re-fetch the current target segment id. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Fixes: milvus-io#30517 Signed-off-by: yangxuan <xuan.yang@zilliz.com>
This PR mainly improve two items: 1. Target observer should refresh loading status during init time. An uninitialized loading status blocks search/query. Currently, the target observer refreshes every 10 seconds, i.e. we'd need to wait for 10s for no reason. That's also the reason why we constantly see false log "collection unloaded" upon mixcoord restarts. 2. Delete session when service is stopped. So that the new service doesn't need to wait for the previous session to expire (~10s). Item 1 is the major improvement of this PR, which should speed up init time by 10s. Item 2 is not a big concern in most cases as coordinators usually shut down after stop(). In those cases, coordinator restart triggers serverID change which further triggers an existing logic that deletes expired session. This PR only fixes rare cases where serverID doesn't change. integration test: `go test -tags dynamic -v -coverprofile=profile.out -covermode=atomic tests/integration/coordrecovery/coord_recovery_test.go -timeout=20m` Performance after the change: Average init time of coordinators: 10s Hardware: M2 Pro Test setup: 1000 collections with 1000 rows (dim=128) per collection. issue: milvus-io#29409 Signed-off-by: yiwangdr <yiwangdr@gmail.com>
Fixes: milvus-io#30507 Signed-off-by: yangxuan <xuan.yang@zilliz.com>
/kind enhancement Signed-off-by: wayblink <anyang.wang@zilliz.com>
…ilvus-io#30504) issue: milvus-io#30358 and milvus-io#30491 Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Data write through rawkv API may pollute tikv data. It should be disallowed. We will add this check to all repos that involves metadata access. In the longer term, we should have a metadata service that implements access control. relate: milvus-io#30029 Signed-off-by: yiwangdr <yiwangdr@gmail.com>
issue: milvus-io#29507 Removing deprecated code from integration test lib Signed-off-by: yiwangdr <yiwangdr@gmail.com>
…io#30555) See also milvus-io#28817 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
relate: milvus-io#30404 --------- Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
) related: milvus-io#29883 Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Related to milvus-io#27675 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…lvus-io#30188) issue: milvus-io#30039 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com>
…vus-io#30549) issue: milvus-io#29880 Signed-off-by: nico <cheng.yuan@zilliz.com>
…0432) milvus-io#30350 Signed-off-by: luzhang <luzhang@zilliz.com> Co-authored-by: luzhang <luzhang@zilliz.com>
feat: restful support new features 1. search with groupingField milvus-io#25324 2. hybrid search milvus-io#25639 Signed-off-by: PowderLi <min.li@zilliz.com>
…io#30532) issue : milvus-io#30369 Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
…ilvus-io#30479) issue: milvus-io#30295 Signed-off-by: jaime <yun.zhang@zilliz.com>
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
issue has been fixed then remove xfail for testcase Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
milvus-io#30523) See also milvus-io#27606 --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…0603) See also milvus-io#30571 When `compactionExecutor` stops one compaction task, the `stop` method will case `injectDone` called. However in `executeTask` when `compact` method returns error, it shall also invoke `injectDone` as well. That the reason `Unlock of unlocked RWMutex` panicking happened. This PR add sync.Once to make sure that `injectDone` is called only once. We did not remove any of the `injectDone` since removal any of those invocation may cause logic problem. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
milvus-io#28410 Signed-off-by: wayblink <anyang.wang@zilliz.com>
wayblink
force-pushed
the
binlogio
branch
4 times, most recently
from
d26adbc
to
d0d2468
Compare
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.