Skip to content

[feature-wip](unique-key-merge-on-write) update delete bitmap when increamental clone#12364

Merged
dataroaring merged 1 commit intoapache:masterfrom
liaoxin01:pk_clone
Sep 9, 2022
Merged

[feature-wip](unique-key-merge-on-write) update delete bitmap when increamental clone#12364
dataroaring merged 1 commit intoapache:masterfrom
liaoxin01:pk_clone

Conversation

@liaoxin01
Copy link
Copy Markdown
Contributor

@liaoxin01 liaoxin01 commented Sep 6, 2022

Proposed changes

Issue Number: close #xxx

Problem summary

Full clone will copy all delete bitmap from src tablet. Incremental clone can't copy the delete bitmap from src table because of compaction. So we should calculate the delete bitmap using incremental rowset data.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@cambyzju
Copy link
Copy Markdown
Contributor

cambyzju commented Sep 6, 2022

Please add some description or link for this pr.

RETURN_IF_ERROR(calc_delete_bitmap(rowset->rowset_id(), segments, nullptr, delete_bitmap,
cur_version - 1, true));

for (auto iter = delete_bitmap->delete_bitmap.begin();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the DeleteBitmap::merge method directly? As well as in another update_delete_bitmap method?

Copy link
Copy Markdown
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2022

PR approved by anyone and no changes requested.

Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 554ba40 into apache:master Sep 9, 2022
@liaoxin01 liaoxin01 deleted the pk_clone branch February 6, 2024 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants