Skip to content

[VL] Enable weekday & date_from_unix_date Spark functions - #4721

Merged
philo-he merged 2 commits into
apache:mainfrom
philo-he:enable-week_day
Feb 21, 2024
Merged

[VL] Enable weekday & date_from_unix_date Spark functions#4721
philo-he merged 2 commits into
apache:mainfrom
philo-he:enable-week_day

Conversation

@philo-he

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

These two functions have been supported in Velox recently.

How was this patch tested?

Spark UTs.

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI

@philo-he

Copy link
Copy Markdown
Member Author

@rui-mo, please take a review. Thanks!

@philo-he

Copy link
Copy Markdown
Member Author

Also cc @taiyang-li, as there is a line of change for CH backend.

@philo-he
philo-he merged commit 316220f into apache:main Feb 21, 2024
@GlutenPerfBot

Copy link
Copy Markdown
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4721_time.csv log/native_master_02_20_2024_c3614f866_time.csv difference percentage
q1 30.91 34.24 3.330 110.77%
q2 24.63 24.42 -0.204 99.17%
q3 39.35 38.78 -0.572 98.55%
q4 38.34 38.03 -0.311 99.19%
q5 79.10 70.83 -8.272 89.54%
q6 5.98 7.23 1.256 121.02%
q7 93.36 82.68 -10.681 88.56%
q8 94.63 85.38 -9.248 90.23%
q9 118.86 126.15 7.290 106.13%
q10 45.77 43.25 -2.523 94.49%
q11 20.74 20.79 0.048 100.23%
q12 26.37 26.48 0.108 100.41%
q13 45.79 44.88 -0.908 98.02%
q14 15.72 18.88 3.160 120.10%
q15 30.08 29.18 -0.902 97.00%
q16 16.19 15.42 -0.772 95.23%
q17 102.86 102.55 -0.307 99.70%
q18 192.07 149.13 -42.945 77.64%
q19 37.99 12.57 -25.418 33.10%
q20 41.86 26.34 -15.522 62.92%
q21 273.43 225.83 -47.606 82.59%
q22 13.68 13.68 -0.002 99.99%
total 1387.72 1236.72 -150.998 89.12%

yikf added a commit to yikf/incubator-gluten that referenced this pull request Aug 3, 2026
…sSuite

The `weekday` Spark function was fully enabled in apache#4721 (WEEK_DAY ->
Velox `weekday`), whose semantics (Monday=0 ... Sunday=6) already match
Spark's WeekDay. However, the corresponding test was left as `ignore`
with a stale "result mismatched" note that no longer applies. Re-enable
it as a regular test, consistent with the adjacent `dayofweek` test.

Generated-by: Claude claude-opus-4-8

Co-authored-by: TRAE CLI <noreply@bytedance.com>
yikf added a commit to yikf/incubator-gluten that referenced this pull request Aug 3, 2026
…sSuite

The `weekday` Spark function was fully enabled in apache#4721 (WEEK_DAY ->
Velox `weekday`), whose semantics (Monday=0 ... Sunday=6) already match
Spark's WeekDay. However, the corresponding test was left as `ignore`
with a stale "result mismatched" note that no longer applies. Re-enable
it as a regular test, consistent with the adjacent `dayofweek` test.
yikf added a commit to yikf/incubator-gluten that referenced this pull request Aug 3, 2026
…sSuite

The `weekday` Spark function was fully enabled in apache#4721 (WEEK_DAY ->
Velox `weekday`), whose semantics (Monday=0 ... Sunday=6) already match
Spark's WeekDay. However, the corresponding test was left as `ignore`
with a stale "result mismatched" note that no longer applies. Re-enable
it as a regular test, consistent with the adjacent `dayofweek` test.
yikf added a commit to yikf/incubator-gluten that referenced this pull request Aug 3, 2026
…sSuite

The `weekday` Spark function was fully enabled in apache#4721 (WEEK_DAY ->
Velox `weekday`), whose semantics (Monday=0 ... Sunday=6) already match
Spark's WeekDay. However, the corresponding test was left as `ignore`
with a stale "result mismatched" note that no longer applies. Re-enable
it as a regular test, consistent with the adjacent `dayofweek` test.
yikf added a commit to yikf/incubator-gluten that referenced this pull request Aug 4, 2026
…sSuite

The `weekday` Spark function was fully enabled in apache#4721 (WEEK_DAY ->
Velox `weekday`), whose semantics (Monday=0 ... Sunday=6) already match
Spark's WeekDay. However, the corresponding test was left as `ignore`
with a stale "result mismatched" note that no longer applies. Re-enable
it as a regular test, consistent with the adjacent `dayofweek` test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants