Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enhance: Avoid merging insert data when buffering insert msgs #33562

Merged
merged 9 commits into from
Jun 13, 2024
Merged

enhance: Avoid merging insert data when buffering insert msgs #33562

merged 9 commits into from
Jun 13, 2024

Commits on Jun 11, 2024

  1. enhance: Pre-allocate insert buffer capacity for writebuffer 

    See also milvus-io#33561

This PR:
- Adds a new param item for insert buffer chunk size
- Pre-allocate for each insert buffer preventing frequent `growslice`

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    eeaca07 View commit details
    Browse the repository at this point in the history

  2. commit param item change 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    ca7bae6 View commit details
    Browse the repository at this point in the history

  3. Avoid use next batch when yield insert 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c114a68 View commit details
    Browse the repository at this point in the history

  4. Use zero copy when buffering insert 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    9490ac6 View commit details
    Browse the repository at this point in the history

  5. Fix lint issue 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    cdbf830 View commit details
    Browse the repository at this point in the history

  6. Filter nil insert data 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    53cef9a View commit details
    Browse the repository at this point in the history

  7. Fix unittest 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    be491bf View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Fix sync empty insert pack panic 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    c1f08be View commit details
    Browse the repository at this point in the history

  2. Fix indexnode ut schema 

    Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
    @congqixia
    congqixia committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    2605393 View commit details
    Browse the repository at this point in the history