Enable minute/second(timestamp_ntz) native execution in Velox backend.#12295
Enable minute/second(timestamp_ntz) native execution in Velox backend.#12295Mariamalmesfer wants to merge 1 commit into
Conversation
|
Run Gluten Clickhouse CI on x86 |
|
CI failure is expected — depends on facebookincubator/velox#17759 which is not yet in the Velox daily build used by CI. |
fafe18c to
44da563
Compare
|
Run Gluten Clickhouse CI on x86 |
44da563 to
456640e
Compare
|
Run Gluten Clickhouse CI on x86 |
456640e to
0f35396
Compare
|
Run Gluten Clickhouse CI on x86 |
0f35396 to
467e7dc
Compare
|
Run Gluten Clickhouse CI on x86 |
There was a problem hiding this comment.
Pull request overview
This PR updates Gluten’s Spark-plan validation logic so minute(timestamp_ntz) and second(timestamp_ntz) projections can be treated as natively supported by the Velox backend (avoiding Spark fallback), and adds Velox backend validation coverage for the new behavior.
Changes:
- Extend the TimestampNTZ validation exception in
Validatorsto recognizeMinuteandSecondexpressions (in addition toHour) operating ontimestamp_ntz. - Add Velox backend suite assertions that
minute(ts)andsecond(ts)overtimestamp_ntzproduce a nativeProjectExecTransformerplan.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| gluten-substrait/src/main/scala/org/apache/gluten/extension/columnar/validator/Validators.scala | Expands the NTZ “extract” allowance in ProjectExec from hour-only to hour/minute/second. |
| backends-velox/src/test/scala/org/apache/gluten/functions/DateFunctionsValidateSuite.scala | Adds validation tests ensuring minute(timestamp_ntz) and second(timestamp_ntz) are planned natively (no fallback). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
467e7dc to
488a06e
Compare
|
Run Gluten Clickhouse CI on x86 |
Enable minute(timestamp_ntz) and second(timestamp_ntz) to run natively in the Velox backend instead of falling back to Spark.
Velox PR: facebookincubator/velox#17759