Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: support null in c data_datacodec and load null value #32183

Merged
merged 5 commits into from
Jul 23, 2024

Conversation

smellthemoon
Copy link
Contributor

@smellthemoon smellthemoon commented Apr 11, 2024

  1. support read and write null in segcore
    will store valid_data(use uint8_t type to save memory) in fieldData.
  2. support load null
    binlog reader read and write data into column(sealed segment), insertRecord(growing segment). In sealed segment, store valid_data directly. In growing segment, considering prior implementation and easy code reading, it covert uint8_t to fbvector, which may optimize in future.
  3. retrieve valid_data.
    parse valid_data in search/query.
    [Feature]: Support Null and default value #31728

@sre-ci-robot sre-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines. label Apr 11, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Apr 11, 2024
Copy link
Contributor

mergify bot commented Apr 11, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 11, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 26, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 26, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 26, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 26, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 28, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 28, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

1 similar comment
Copy link
Contributor

mergify bot commented Apr 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Apr 29, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented May 6, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented May 6, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented May 6, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented May 6, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

@sre-ci-robot sre-ci-robot added the area/dependency Pull requests that update a dependency file label May 7, 2024
Copy link
Contributor

mergify bot commented May 7, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented May 7, 2024

@smellthemoon ut workflow job failed, comment rerun ut can trigger the job again.

Copy link
Contributor

mergify bot commented Jul 19, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@smellthemoon smellthemoon force-pushed the support-payload-c branch 6 times, most recently from 4ab36e2 to 0167033 Compare July 22, 2024 06:06
Copy link
Contributor

mergify bot commented Jul 22, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

lixinguo added 5 commits July 23, 2024 12:44
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Copy link

codecov bot commented Jul 23, 2024

Codecov Report

Attention: Patch coverage is 75.83187% with 138 lines in your changes missing coverage. Please review.

Project coverage is 80.92%. Comparing base (1f55061) to head (a4835f6).
Report is 5 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #32183      +/-   ##
==========================================
- Coverage   80.96%   80.92%   -0.04%     
==========================================
  Files        1185     1180       -5     
  Lines      142715   143366     +651     
==========================================
+ Hits       115544   116018     +474     
- Misses      22760    22917     +157     
- Partials     4411     4431      +20     
Files Coverage Δ
internal/core/src/common/FieldData.h 100.00% <100.00%> (ø)
internal/core/src/common/FieldMeta.h 94.64% <100.00%> (+0.76%) ⬆️
internal/core/src/common/Schema.h 100.00% <100.00%> (ø)
internal/core/src/common/Vector.h 100.00% <100.00%> (ø)
internal/core/src/index/StringIndexMarisa.cpp 88.81% <100.00%> (-0.14%) ⬇️
internal/core/src/query/SearchOnGrowing.cpp 100.00% <100.00%> (ø)
...nal/core/src/query/groupby/SearchGroupByOperator.h 85.00% <100.00%> (-0.25%) ⬇️
internal/core/src/segcore/ConcurrentVector.h 96.96% <ø> (-1.51%) ⬇️
internal/core/src/segcore/Utils.h 100.00% <ø> (ø)
internal/core/src/segcore/segment_c.cpp 65.72% <100.00%> (-0.59%) ⬇️
... and 28 more

... and 212 files with indirect coverage changes

@sunby
Copy link
Contributor

sunby commented Jul 23, 2024

/lgtm
btw, I plan to add a null bitmap concatenater to merge bitmaps. after that you can replace FixedVector with it.

@congqixia
Copy link
Contributor

/approve

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia, smellthemoon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mergify mergify bot added the ci-passed label Jul 23, 2024
@sre-ci-robot sre-ci-robot merged commit 5616b7e into milvus-io:master Jul 23, 2024
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved area/compilation area/dependency Pull requests that update a dependency file ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/XXL Denotes a PR that changes 1000+ lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants