Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: return growing segment when sealed #32460

Merged
merged 1 commit into from
Apr 19, 2024
Merged

fix: return growing segment when sealed #32460

merged 1 commit into from
Apr 19, 2024

Conversation

chyezh
Copy link
Contributor

@chyezh chyezh commented Apr 19, 2024 •
edited
Loading

issue: #32435,#32453

@chyezh
fix: return growing segment when sealed
561470a 
Signed-off-by: chyezh <chyezh@outlook.com>
@sre-ci-robot sre-ci-robot added the size/XS Denotes a PR that changes 0-9 lines. label Apr 19, 2024
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chyezh
To complete the pull request process, please assign yanliang567 after the PR has been reviewed.
You can assign the PR to them by writing /assign @yanliang567 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jaime0815 jaime0815 merged commit bcf6c48 into milvus-io:lru-dev Apr 19, 2024
6 of 8 checks passed
@chyezh chyezh deleted the fix_get_sealed_segment_with_growing branch April 19, 2024 07:04
@chyezh chyezh mentioned this pull request Apr 22, 2024
Closed
sunby pushed a commit to sunby/milvus that referenced this pull request Apr 22, 2024
@chyezh @sunby
Fix partital release (milvus-io#31866)
4c583a5 
Signed-off-by: chyezh <chyezh@outlook.com>

Add metric for lru and fix lost delete data when enable lazy load  (milvus-io#31868)

Signed-off-by: chyezh <chyezh@outlook.com>

feat: Support stream reduce v1 (milvus-io#31873)

related: milvus-io#31410

---------

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

Change do wait lru dev (milvus-io#31878)

Signed-off-by: sunby <sunbingyi1992@gmail.com>

enhance: add config for disk cache (milvus-io#31881)

fix config not initialized (milvus-io#31890)

Signed-off-by: sunby <sunbingyi1992@gmail.com>

fix error handle in search (milvus-io#31895)

Signed-off-by: sunby <sunbingyi1992@gmail.com>

fix: thread safe vector (milvus-io#31898)

fix: insert record cannot reinsert (milvus-io#31900)

enhance: cancel concurrency restrict for stream reduce and add metrics (milvus-io#31892)

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

fix: bit set (milvus-io#31905)

fix bitset clear to reset (milvus-io#31908)

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

Fix 0404 lru dev (milvus-io#31914)

fix:
1. sealed_segment num_rows reset to std::null opt
2. sealed_segment lazy_load reset to true after evicting to avoid
shortcut

---------

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

fix possible block due to unpin fifo activating principle (milvus-io#31924)

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

Add lru reloader lru dev (milvus-io#31952)

Signed-off-by: sunby <sunbingyi1992@gmail.com>

fix query limit (milvus-io#32060)

Signed-off-by: sunby <sunbingyi1992@gmail.com>

fix: lru cache lost delete and wrong mem size (milvus-io#32072)

issue: milvus-io#30361

Signed-off-by: chyezh <chyezh@outlook.com>

enhance: add more metrics for cache and search (milvus-io#31777) (milvus-io#32097)

issue: milvus-io#30931

Signed-off-by: chyezh <chyezh@outlook.com>

fix:panic due to empty search result when stream reducing(milvus-io#32009) (milvus-io#32083)

related: milvus-io#32009

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

fix: sealed segment may not exist when throw (milvus-io#32098)

issue: milvus-io#30361

Signed-off-by: chyezh <chyezh@outlook.com>

Major compaction 1st edition (milvus-io#31804) (milvus-io#32116)

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: chasingegg <chao.gao@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>

fix: inconsistent between state lock and load state (milvus-io#32171)

issue: milvus-io#30361

Signed-off-by: chyezh <chyezh@outlook.com>

enhance: Throw error instead of crash when index cannot be built (milvus-io#31844)

issue: milvus-io#27589

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>

(cherry picked from commit 1b76766)
Signed-off-by: jaime <yun.zhang@zilliz.com>

update knowhere to support clustering (milvus-io#32188)

Signed-off-by: chasingegg <chao.gao@zilliz.com>

fix: segment release is not sync with cache (milvus-io#32212)

issue: milvus-io#32206

Signed-off-by: chyezh <chyezh@outlook.com>

fix: incorrect pinCount resulting unexpected eviction(milvus-io#32136) (milvus-io#32238)

related: milvus-io#32136

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

fix: possible panic when stream reducing (milvus-io#32247)

related: milvus-io#32009

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

enhance: [lru-dev] add the related data size for the read apis (milvus-io#32274)

cherry-pick: milvus-io#31816

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>

add debug log (milvus-io#32303)

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>

Refine code for analyze task scheduler (milvus-io#32122)

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>

fix: memory leak on stream reduce (milvus-io#32345)

related: milvus-io#32304

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

feat: adding cache stats support (milvus-io#32344)

See milvus-io#32067

Signed-off-by: Ted Xu <ted.xu@zilliz.com>

Fix bug for version (milvus-io#32363)

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>

fix: remove sub entity in load delta log, update entity num in segment itself (milvus-io#32350)

issue: milvus-io#30361

Signed-off-by: chyezh <chyezh@outlook.com>

fix: clear data when loading failure (milvus-io#32370)

issue: milvus-io#30361

Signed-off-by: chyezh <chyezh@outlook.com>

fix: stream reduce memory leak for failing to release stream reducer(milvus-io#32345) (milvus-io#32381)

related: milvus-io#32345

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

Keep InProgress state when getting task state is init (milvus-io#32394)

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>

add log for search failed (milvus-io#32367)

related: milvus-io#32136

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

enable asan by default (milvus-io#32423)

Signed-off-by: sunby <sunbingyi1992@gmail.com>

Major compaction refactoring (milvus-io#32149)

Signed-off-by: wayblink <anyang.wang@zilliz.com>

Lru dev debug (milvus-io#32414)

Co-authored-by: wayblink <anyang.wang@zilliz.com>

fix: protect loadInfo with atomic, remove rlock at cache to avoid dead lock (milvus-io#32436)

issue: milvus-io#32435

Signed-off-by: chyezh <chyezh@outlook.com>

fix: use Get but not GetBy of SegmentManager (milvus-io#32438)

issue: milvus-io#32435

Signed-off-by: chyezh <chyezh@outlook.com>

fix: return growing segment when sealed (milvus-io#32460)

issue: milvus-io#32435

Signed-off-by: chyezh <chyezh@outlook.com>

enhance: add request resource for lru loading process(milvus-io#32205) (milvus-io#32452)

related: milvus-io#32205

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

fix: unexpected deleted index files when lazy loading(milvus-io#32136) (milvus-io#32469)

related: milvus-io#32136

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>

fix: reference count leak cause release blocked (milvus-io#32465)

issue: milvus-io#32379

Signed-off-by: chyezh <chyezh@outlook.com>

Fix compaction fail (milvus-io#32473)

Signed-off-by: wayblink <anyang.wang@zilliz.com>
@chyezh
Copy link
Contributor Author

chyezh commented Apr 25, 2024

master pr: #32567

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aoiasd aoiasd Awaiting requested review from aoiasd

@yah01 yah01 Awaiting requested review from yah01

Assignees
No one assigned
Labels
size/XS Denotes a PR that changes 0-9 lines.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@chyezh @sre-ci-robot @jaime0815