Skip to content

[enhancement](merge-on-write) add async publish task when version is discontinuous for merge on write table when clone#21025

Merged
yiguolei merged 6 commits intoapache:masterfrom
liaoxin01:optimize_clone
Jun 22, 2023
Merged

[enhancement](merge-on-write) add async publish task when version is discontinuous for merge on write table when clone#21025
yiguolei merged 6 commits intoapache:masterfrom
liaoxin01:optimize_clone

Conversation

@liaoxin01
Copy link
Contributor

@liaoxin01 liaoxin01 commented Jun 20, 2023 •
edited
Loading

Proposed changes

Issue Number: close #xxx

version discontinuity may occur when clone. To deal with this case, add async publish task when version is discontinuous.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Contributor

github-actions bot commented Jun 20, 2023

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Jun 20, 2023

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Contributor

github-actions bot commented Jun 20, 2023

clang-tidy review says "All clean, LGTM! 👍"

@xiaokang
Copy link
Contributor

xiaokang commented Jun 21, 2023

run buildall

@liaoxin01 liaoxin01 force-pushed the optimize_clone branch 2 times, most recently from 64584c7 to b14cfb6 Compare June 21, 2023 08:18
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

dataroaring
dataroaring previously approved these changes Jun 21, 2023
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring
Copy link
Contributor

dataroaring commented Jun 21, 2023

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 21, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 21, 2023
@dataroaring
Copy link
Contributor

dataroaring commented Jun 21, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

@liaoxin01
Copy link
Contributor Author

liaoxin01 commented Jun 21, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

@liaoxin01 liaoxin01 changed the title [enhancement](merge-on-write) add aync publish task when version is discontinous for merge on write table when clone [enhancement](merge-on-write) add async publish task when version is discontinous for merge on write table when clone Jun 21, 2023
@liaoxin01 liaoxin01 changed the title [enhancement](merge-on-write) add async publish task when version is discontinous for merge on write table when clone [enhancement](merge-on-write) add async publish task when version is discontinuous for merge on write table when clone Jun 21, 2023
@liaoxin01
Copy link
Contributor Author

liaoxin01 commented Jun 21, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

@liaoxin01
Copy link
Contributor Author

liaoxin01 commented Jun 21, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

clang-tidy review says "All clean, LGTM! 👍"

zhannngchen
zhannngchen approved these changes Jun 21, 2023
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 21, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2023

PR approved by at least one committer and no changes requested.

dataroaring
dataroaring approved these changes Jun 21, 2023
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 22, 2023

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.1 seconds
stream load tsv: 460 seconds loaded 74807831229 Bytes, about 155 MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select: 68.6 seconds inserted 10000000 Rows, about 145K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230622024702_clickbench_pr_166487.html

@yiguolei yiguolei merged commit 691a988 into apache:master Jun 22, 2023
@yiguolei yiguolei added the dev/2.0.0 2.0.0 release label Jun 22, 2023
xiaokang pushed a commit that referenced this pull request Jun 25, 2023
@liaoxin01 @xiaokang
[enhancement](merge-on-write) add async publish task when version is …
e599711 
…discontinuous for merge on write table when clone (#21025)

version discontinuity may occur when clone. To deal with this case, add async publish task when version is discontinuous.
@liaoxin01 liaoxin01 mentioned this pull request Jun 29, 2023
Merged
@zhannngchen zhannngchen added dev/2.0-beta and removed dev/2.0.0 2.0.0 release labels Jun 29, 2023
@liaoxin01 liaoxin01 deleted the optimize_clone branch February 6, 2024 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhannngchen zhannngchen zhannngchen approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0-beta-merged maybe-later reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@liaoxin01 @xiaokang @dataroaring @hello-stephen @zhannngchen @yiguolei