Skip to content

Conversation

@kaijchen
Copy link
Member

Proposed changes

Do not ignore status in LoadStreamWriter::close()

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@kaijchen
Copy link
Member Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@kaijchen
Copy link
Member Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.34% (8023/22079)
Line Coverage: 28.64% (64432/224975)
Region Coverage: 27.55% (33468/121492)
Branch Coverage: 24.21% (17119/70714)
Coverage Report: http://coverage.selectdb-in.cc/coverage/738e4375a8a302e9e97d8fec70d20a83b9ec03c1_738e4375a8a302e9e97d8fec70d20a83b9ec03c1/report/index.html

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.84 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162094080 Bytes

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.33% (8021/22079)
Line Coverage: 28.63% (64420/224975)
Region Coverage: 27.54% (33458/121492)
Branch Coverage: 24.21% (17122/70714)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f518d695ffd8e4d4c13353fc6f03a929fdbda17c_f518d695ffd8e4d4c13353fc6f03a929fdbda17c/report/index.html

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.44 seconds
stream load tsv: 550 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162197814 Bytes

@kaijchen kaijchen force-pushed the fix-load-stream-writer branch from f518d69 to 8e1ca6b Compare October 6, 2023 23:59
@kaijchen
Copy link
Member Author

kaijchen commented Oct 7, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Oct 7, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.31% (8150/22447)
Line Coverage: 28.44% (65196/229240)
Region Coverage: 27.38% (33765/123335)
Branch Coverage: 24.04% (17223/71630)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8e1ca6b2cb840310941cc62285e28c4fb5601db0_8e1ca6b2cb840310941cc62285e28c4fb5601db0/report/index.html

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.33 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162156676 Bytes

@kaijchen
Copy link
Member Author

kaijchen commented Oct 7, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Oct 7, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.30% (8148/22447)
Line Coverage: 28.43% (65176/229254)
Region Coverage: 27.37% (33757/123335)
Branch Coverage: 24.04% (17218/71630)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8e192d25f0aa7e4f015894b890e539a3d57fd18a_8e192d25f0aa7e4f015894b890e539a3d57fd18a/report/index.html

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.54 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162141709 Bytes

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@Hastyshell Hastyshell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 8, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Oct 8, 2023

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 8, 2023

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 73f632a into apache:master Oct 11, 2023
@kaijchen kaijchen deleted the fix-load-stream-writer branch October 12, 2023 03:57
dutyu pushed a commit to dutyu/doris that referenced this pull request Oct 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants