Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from milvus-io:master #8

Open
wants to merge 1,687 commits into
base: master
Choose a base branch
from
Open

Conversation

pull[bot]
Copy link

@pull pull bot commented May 7, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added the ⤵️ pull label May 7, 2024
congqixia and others added 29 commits June 12, 2024 22:29
See also #33789

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #33561

This PR:
- Use zero copy when buffering insert messages
- Make `storage.InsertCodec` support serialize multiple insert data
chunk into same batch binlog files

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.

issue: #33142

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
See also #33716 #33823

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
add test case for #33681

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Add commnets for meta table in internal/rootcoord/meta_table.go.

Signed-off-by: ArenaSu <704427617@qq.com>
Add comments for event manager(internal/datanode/event_manager.go).

Signed-off-by: ArenaSu <704427617@qq.com>
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
See also #33823

`EvictCacheValueByFormat` may be block by on going `CASCacheValue` and
cause possible deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
issue: #33432

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
…redict (#33792)

issue: #33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #33839

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
Move compaction executor to compaction pacakge.

issue: #32451

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
issue: #30857

---------

Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Signed-off-by: nico <cheng.yuan@zilliz.com>
See also: #33819

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
)

#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
#32900

---------

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Due to the removal of injection and syncSegments from the compaction, we
need to ensure that no compaction is successfully executed during the
rolling upgrade. This PR renames Compaction to CompactionV2, with the
following effects:
- New datacoord + old datanode: Utilizes the CompactionV2 interface,
resulting in the datanode error "CompactionV2 not implemented," causing
compaction to fail;
- Old datacoord + new datanode: Utilizes the CompactionV1 interface,
resulting in the datanode error "CompactionV1 not implemented," causing
compaction to fail.

issue: #32809

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
/kind enhancement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
issue: #32707

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #33540 
1. gorwing L0 segments is invisible to datacoord.
2. flushed L0 segments need to clean by datacoord.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
See also #33561 #33562

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
weiliu1031 and others added 30 commits July 10, 2024 09:51
issue: #34540

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
See also #34519

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…collection creation (#34403)

issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Related to #34508

The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…trol (#34558)

issue: #34557

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
When the main dispatcher has not yet consumed data, curTs is 0. During
this time, merging dispatchers should not be allowed; otherwise, the
data of the solo dispatcher will be skipped.

issue: #34255

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
issue: # #34545

Print warn log instead of check health fail if orphan channel cp meta is
found in health check request.

Signed-off-by: jaime <yun.zhang@zilliz.com>
issue: #33285

- add balancer implementation
- add channel count fair balance policy
- add channel assignment discover grpc service

Signed-off-by: chyezh <chyezh@outlook.com>
issue: #34533

Signed-off-by: jaime <yun.zhang@zilliz.com>
See also #34574

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…34103)

See also: #31059

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #33419 #34426

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
related with: #29258

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
#34527

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Fix L0 compaction task recover unable to set segment not isCompacting

See also: #34460

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #34332

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
This bug caused failed L0 compaction tasks never end.

See also: #34460

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #34545

Signed-off-by: jaime <yun.zhang@zilliz.com>
…ss if crash (#34513)

#34512

Signed-off-by: wayblink <anyang.wang@zilliz.com>
…4576)

relate: #34578

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
related: #33544

mainly changes in three aspects:

1. enable setting group_size for group by function
2. separate normal reduce and group by reduce
3. eleminate uncessary padding in search result for reducing

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
github.com/gogo/protobuf is deprecated and could be error prune after
upgrade protobuf message to v2.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
- issue: #34614

Signed-off-by: SimFG <bang.fu@zilliz.com>
See also #28817

Add paramitem for segment flush interval

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
…rt (#34649)

issue: #34648

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
…ector case (#34618)

If `is_all_data_type` is true, the case will add float32, f16, bf16, and
sparse vectors, but the created indexes are all `flat` indexes by
default. The sparse type cannot create a flat index. Fix the test code
to create a `SPARSE_INVERTED_INDEX` index for the sparse vector when
is_all_data_type is true

Signed-off-by: elstic <hao.wang@zilliz.com>
#33704

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
#34427

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
#34603

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet