-
Notifications
You must be signed in to change notification settings - Fork 219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix array_transform to not recompute the argument #10015
Fix array_transform to not recompute the argument #10015
Conversation
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
build |
1 similar comment
build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. other than a nit
val explodedTable = withResource(GpuProjectExec.project(inputBatch, boundIntermediate)) { | ||
intermediateBatch => | ||
withResource(GpuColumnVector.combineColumns(intermediateBatch, argColumn)) { | ||
projectedBatch => | ||
withResource(GpuColumnVector.from(projectedBatch)) { projectedTable => | ||
projectedTable.explodePosition(boundIntermediate.length) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: could this and should this be refactored to a private method? It can be reused by passing the method to run explodePosition
or explode
@@ -895,6 +895,22 @@ public static ColumnarBatch combineColumns(ColumnarBatch ... batches) { | |||
return incRefCounts(ret); | |||
} | |||
|
|||
public static ColumnarBatch combineColumns(ColumnarBatch cb, GpuColumnVector ... vectors) { | |||
final int numRows = cb.numRows(); | |||
ArrayList<ColumnVector> columns = new ArrayList<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: We know exactly how big the array needs to be and can hint this here.
ArrayList<ColumnVector> columns = new ArrayList<>(); | |
ArrayList<ColumnVector> columns = new ArrayList<>(cb.numCols() + vectors.length); |
build |
@razajafri and @jlowe please take another look |
This fixes #9472
Thanks to @razajafri for doing the majority of the debugging on this.