-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Export constant vectors as REE arrow schema #7327
Conversation
✅ Deploy Preview for meta-velox canceled.
|
This pull request was exported from Phabricator. Differential Revision: D50819081 |
88cc29e
to
a51c837
Compare
Summary: This is the first PR to add support for constant vectors in Velox through the Arrow bridge. The PR maps the types and adds appropriate tests. The next one will map the actual data. Differential Revision: D50819081
This pull request was exported from Phabricator. Differential Revision: D50819081 |
Summary: This is the first PR to add support for constant vectors in Velox through the Arrow bridge. The PR maps the types and adds appropriate tests. The next one will map the actual data. Differential Revision: D50819081
a51c837
to
ba07fe4
Compare
This pull request was exported from Phabricator. Differential Revision: D50819081 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change looks good. Two minor comments that can be addressed in the follow-up PR.
@@ -91,6 +96,59 @@ class ArrowBridgeSchemaExportTest : public testing::Test { | |||
} | |||
} | |||
|
|||
void testConstant(const TypePtr& type, const char* arrowFormat) { | |||
ArrowSchema arrowSchema; | |||
const bool isScalar = type->size() == 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit const bool isScalar = (type->size() == 0);
@@ -644,6 +663,26 @@ void exportToArrow(const VectorPtr& vec, ArrowSchema& arrowSchema) { | |||
arrowSchema.dictionary = bridgeHolder->dictionary.get(); | |||
exportToArrow(vec->valueVector(), *arrowSchema.dictionary); | |||
|
|||
} else if (vec->encoding() == VectorEncoding::Simple::CONSTANT) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It helps to add a link to the spec https://arrow.apache.org/docs/format/Columnar.html#run-end-encoded-layout
ba07fe4
to
444a1ef
Compare
Summary: This is the first PR to add support for constant vectors in Velox through the Arrow bridge. The PR maps the types and adds appropriate tests. The next one will map the actual data. Reviewed By: mbasmanova Differential Revision: D50819081
Summary: This is the first PR to add support for constant vectors in Velox through the Arrow bridge. The PR maps the types and adds appropriate tests. The next one will map the actual data. Reviewed By: mbasmanova Differential Revision: D50819081
This pull request was exported from Phabricator. Differential Revision: D50819081 |
444a1ef
to
02eb286
Compare
This pull request was exported from Phabricator. Differential Revision: D50819081 |
Thanks @majetideepak , comments addressed. |
This pull request has been merged in d8d362c. |
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Summary:
This is the first PR to add support for constant vectors in Velox
through the Arrow bridge. The PR maps the types and adds appropriate tests. The
next one will map the actual data.
Differential Revision: D50819081