Remove dynamic cast from FlatVector::copyValuesAndNulls #1316
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
mbasmanova
force-pushed
the
remove-dynamic-cast
branch
from
b5f0368
to
c0a73db
Compare
|
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
mbasmanova
force-pushed
the
remove-dynamic-cast
branch
from
c0a73db
to
a2c67a3
Compare
|
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
mbasmanova
added a commit
to mbasmanova/velox-1
that referenced
this pull request
Apr 2, 2022
Summary: - Remove extra copying into RowContainer. - Use TreeOfLosers for merging. - Copy rows to output in bulk. This version is much faster than previous. Merging 7 streams of 80M rows each on 1 integer key is 27s vs original 70s and on 2 keys is 32s vs original 290s. These numbers include optimizations for FlatVector::copyValuesAndNulls (facebookincubator#1316) and BaseVector::compare (facebookincubator#1317) 1 flat int key <img width="1047" alt="Screen Shot 2022-03-31 at 7 03 18 AM" src="https://user-images.githubusercontent.com/27965151/161041090-1750a8c0-9f82-4bb1-9e96-5bc2d4929691.png"> 2 keys: flat and const: <img width="1044" alt="Screen Shot 2022-03-31 at 7 03 44 AM" src="https://user-images.githubusercontent.com/27965151/161041104-c38a923d-96dc-4f05-bb2f-a42c2f409b62.png"> Pull Request resolved: facebookincubator#1318 Reviewed By: oerling Differential Revision: D35274106 Pulled By: mbasmanova fbshipit-source-id: 3f53545be3411347a78609fb9d45ceeb5acea904
mbasmanova
added a commit
to mbasmanova/velox-1
that referenced
this pull request
Apr 4, 2022
Summary: - Remove extra copying into RowContainer. - Use TreeOfLosers for merging. - Copy rows to output in bulk. This version is much faster than previous. Merging 7 streams of 80M rows each on 1 integer key is 27s vs original 70s and on 2 keys is 32s vs original 290s. These numbers include optimizations for FlatVector::copyValuesAndNulls (facebookincubator#1316) and BaseVector::compare (facebookincubator#1317) 1 flat int key <img width="1047" alt="Screen Shot 2022-03-31 at 7 03 18 AM" src="https://user-images.githubusercontent.com/27965151/161041090-1750a8c0-9f82-4bb1-9e96-5bc2d4929691.png"> 2 keys: flat and const: <img width="1044" alt="Screen Shot 2022-03-31 at 7 03 44 AM" src="https://user-images.githubusercontent.com/27965151/161041104-c38a923d-96dc-4f05-bb2f-a42c2f409b62.png"> Pull Request resolved: facebookincubator#1318 Reviewed By: oerling Differential Revision: D35274106 Pulled By: mbasmanova fbshipit-source-id: b5134883eb72f9e15f35127e4731fc23e01a3f86
mbasmanova
added a commit
to mbasmanova/velox-1
that referenced
this pull request
Apr 4, 2022
Summary: - Remove extra copying into RowContainer. - Use TreeOfLosers for merging. - Copy rows to output in bulk. This version is much faster than previous. Merging 7 streams of 80M rows each on 1 integer key is 27s vs original 70s and on 2 keys is 32s vs original 290s. These numbers include optimizations for FlatVector::copyValuesAndNulls (facebookincubator#1316) and BaseVector::compare (facebookincubator#1317) 1 flat int key <img width="1047" alt="Screen Shot 2022-03-31 at 7 03 18 AM" src="https://user-images.githubusercontent.com/27965151/161041090-1750a8c0-9f82-4bb1-9e96-5bc2d4929691.png"> 2 keys: flat and const: <img width="1044" alt="Screen Shot 2022-03-31 at 7 03 44 AM" src="https://user-images.githubusercontent.com/27965151/161041104-c38a923d-96dc-4f05-bb2f-a42c2f409b62.png"> Pull Request resolved: facebookincubator#1318 Reviewed By: oerling Differential Revision: D35274106 Pulled By: mbasmanova fbshipit-source-id: 45b626bc88baf1ecf86b37110ca663d6839a41fc
facebook-github-bot
pushed a commit
that referenced
this pull request
Apr 4, 2022
Summary: - Remove extra copying into RowContainer. - Use TreeOfLosers for merging. - Copy rows to output in bulk. This version is much faster than previous. Merging 7 streams of 80M rows each on 1 integer key is 27s vs original 70s and on 2 keys is 32s vs original 290s. These numbers include optimizations for FlatVector::copyValuesAndNulls (#1316) and BaseVector::compare (#1317) 1 flat int key <img width="1047" alt="Screen Shot 2022-03-31 at 7 03 18 AM" src="https://user-images.githubusercontent.com/27965151/161041090-1750a8c0-9f82-4bb1-9e96-5bc2d4929691.png"> 2 keys: flat and const: <img width="1044" alt="Screen Shot 2022-03-31 at 7 03 44 AM" src="https://user-images.githubusercontent.com/27965151/161041104-c38a923d-96dc-4f05-bb2f-a42c2f409b62.png"> Pull Request resolved: #1318 Reviewed By: oerling Differential Revision: D35274106 Pulled By: mbasmanova fbshipit-source-id: 4f05cd53edbf76f03064edcc3bb64777d65c84e9
shiyu-bytedance
pushed a commit
to shiyu-bytedance/velox-1
that referenced
this pull request
Aug 18, 2022
…bator#1316) Summary: Dynamic casts in FlatVector::copyValuesAndNulls showed up on the profile of a local merge operator. Replacing dynamic_cast with static_cast helped reduce the total time of merging 7 streams of 80M rows from 83s to 45s. Profile before: <img width="1086" alt="Screen Shot 2022-03-30 at 2 29 04 PM" src="https://user-images.githubusercontent.com/27965151/160908836-ccc0b68e-41ea-4516-8b17-8894f0abe96a.png"> and after: <img width="1112" alt="Screen Shot 2022-03-30 at 2 42 26 PM" src="https://user-images.githubusercontent.com/27965151/160908824-c4bdeaae-69e1-4247-b847-7f2d96783e79.png"> Pull Request resolved: facebookincubator#1316 Reviewed By: kgpai Differential Revision: D35258498 Pulled By: mbasmanova fbshipit-source-id: c051ef69b6891724f3f7ff5bf094d5beb9f2e5d1
shiyu-bytedance
pushed a commit
to shiyu-bytedance/velox-1
that referenced
this pull request
Aug 18, 2022
Summary: - Remove extra copying into RowContainer. - Use TreeOfLosers for merging. - Copy rows to output in bulk. This version is much faster than previous. Merging 7 streams of 80M rows each on 1 integer key is 27s vs original 70s and on 2 keys is 32s vs original 290s. These numbers include optimizations for FlatVector::copyValuesAndNulls (facebookincubator#1316) and BaseVector::compare (facebookincubator#1317) 1 flat int key <img width="1047" alt="Screen Shot 2022-03-31 at 7 03 18 AM" src="https://user-images.githubusercontent.com/27965151/161041090-1750a8c0-9f82-4bb1-9e96-5bc2d4929691.png"> 2 keys: flat and const: <img width="1044" alt="Screen Shot 2022-03-31 at 7 03 44 AM" src="https://user-images.githubusercontent.com/27965151/161041104-c38a923d-96dc-4f05-bb2f-a42c2f409b62.png"> Pull Request resolved: facebookincubator#1318 Reviewed By: oerling Differential Revision: D35274106 Pulled By: mbasmanova fbshipit-source-id: 4f05cd53edbf76f03064edcc3bb64777d65c84e9
marin-ma
pushed a commit
to marin-ma/velox-oap
that referenced
this pull request
Dec 15, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Dynamic casts in FlatVector::copyValuesAndNulls showed up on the profile of a local merge operator. Replacing dynamic_cast with static_cast helped reduce the total time of merging 7 streams of 80M rows from 83s to 45s.
Profile before:
and after: