-
Notifications
You must be signed in to change notification settings - Fork 390
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[VL] Enable array_repeat & array_except function #4504
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/oap-project/gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
Run Gluten Clickhouse CI |
6de0008
to
058bf1d
Compare
Run Gluten Clickhouse CI |
058bf1d
to
b2022a1
Compare
Run Gluten Clickhouse CI |
b2022a1
to
8e83f4a
Compare
Run Gluten Clickhouse CI |
8e83f4a
to
9115f51
Compare
Run Gluten Clickhouse CI |
9115f51
to
727c796
Compare
Run Gluten Clickhouse CI |
A velox PR is fixing the test issue reported for array_repeat: facebookincubator/velox#8630. �[31m- SPARK-36753: ArrayExcept should handle duplicated Double.NaN and Float.Nan *** FAILED ***�[0m
2024-01-24T07:34:28.7280340Z �[31m Incorrect evaluation: array_except([NaN,1.0], [NaN]), actual: WrappedArray(NaN, 1.0), expected: List(1.0) (GlutenTestsTrait.scala:288)�[0m
|
727c796
to
60bef4e
Compare
Run Gluten Clickhouse CI |
60bef4e
to
9d6376c
Compare
Run Gluten Clickhouse CI |
9d6376c
to
a495429
Compare
Run Gluten Clickhouse CI |
Run Gluten Clickhouse CI |
ENCODE -> EncodeDecodeValidator() | ||
ENCODE -> EncodeDecodeValidator(), | ||
ARRAY_EXCEPT -> DefaultValidator(), | ||
ARRAY_REPEAT -> DefaultValidator() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc @taiyang-li, I directly changed this for CH backend to let these two unsupported functions fall back.
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
What changes were proposed in this pull request?
These functions should have been supported: array_repeat, array_except, arrray_distinct, array_position. Two of them only need few code to get them correctly mapped to velox function. Also added tests to verify they are offoaded.
How was this patch tested?
Spark UT & new test.