Skip to content

[GLUTEN-8319][VL] Support date_format Spark function#8323

Merged
zhli1142015 merged 1 commit intoapache:mainfrom
PHILO-HE:fix-date_format
Jan 13, 2025
Merged

[GLUTEN-8319][VL] Support date_format Spark function#8323
zhli1142015 merged 1 commit intoapache:mainfrom
PHILO-HE:fix-date_format

Conversation

@PHILO-HE
Copy link
Member

@PHILO-HE PHILO-HE commented Dec 24, 2024

What changes were proposed in this pull request?

facebookincubator/velox#11953.

Fix #8319.

How was this patch tested?

Existing Spark test and one new test added.

@github-actions
Copy link

#8319

@PHILO-HE PHILO-HE changed the title [GLUTEN-8319]VL] Support date_format Spark function [GLUTEN-8319][VL] Support date_format Spark function Dec 25, 2024
@PHILO-HE
Copy link
Member Author

@zhli1142015, could you take a look?

Copy link
Contributor

@zhli1142015 zhli1142015 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@zhli1142015 zhli1142015 merged commit f79f052 into apache:main Jan 13, 2025
51 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_01_13_2025_time.csv log/native_master_01_12_2025_d70de0bb52_time.csv difference percentage
q1 15.37 16.26 0.883 105.74%
q2 16.51 16.68 0.173 101.05%
q3 5.62 4.48 -1.136 79.78%
q4 85.99 86.16 0.168 100.20%
q5 14.41 13.35 -1.056 92.67%
q6 5.04 3.72 -1.318 73.83%
q7 5.90 7.89 1.989 133.71%
q8 4.45 6.26 1.809 140.65%
q9 27.45 27.56 0.115 100.42%
q10 12.76 12.51 -0.252 98.03%
q11 43.71 43.39 -0.323 99.26%
q12 2.37 3.45 1.074 145.26%
q13 9.10 8.87 -0.225 97.53%
q14a 60.80 68.35 7.545 112.41%
q14b 56.62 57.33 0.704 101.24%
q15 3.41 3.47 0.056 101.64%
q16 29.42 28.83 -0.589 98.00%
q17 7.28 7.64 0.360 104.95%
q18 10.31 9.87 -0.444 95.69%
q19 3.66 3.69 0.026 100.71%
q20 2.14 2.44 0.298 113.95%
q21 1.54 1.53 -0.013 99.15%
q22 11.26 9.65 -1.609 85.71%
q23a 135.38 140.18 4.803 103.55%
q23b 163.70 162.07 -1.628 99.01%
q24a 99.88 107.39 7.514 107.52%
q24b 96.72 101.46 4.741 104.90%
q25 9.99 6.50 -3.495 65.02%
q26 5.28 4.37 -0.912 82.74%
q27 6.72 5.71 -1.012 84.94%
q28 38.75 39.14 0.387 101.00%
q29 19.51 19.78 0.275 101.41%
q30 8.01 8.40 0.390 104.86%
q31 10.56 10.52 -0.037 99.65%
q32 2.30 2.34 0.036 101.58%
q33 8.81 6.96 -1.858 78.92%
q34 4.51 4.58 0.064 101.42%
q35 10.88 10.10 -0.781 92.82%
q36 5.74 5.81 0.070 101.22%
q37 5.24 5.17 -0.073 98.62%
q38 17.48 18.13 0.651 103.72%
q39a 4.63 4.62 -0.012 99.73%
q39b 5.77 4.98 -0.784 86.41%
q40 5.35 5.81 0.461 108.61%
q41 0.87 1.15 0.280 132.38%
q42 1.19 1.25 0.067 105.64%
q43 4.73 4.53 -0.203 95.71%
q44 11.79 12.87 1.081 109.16%
q45 4.14 4.23 0.088 102.13%
q46 5.46 5.36 -0.101 98.15%
q47 21.32 21.93 0.615 102.88%
q48 6.59 6.55 -0.036 99.46%
q49 10.04 10.65 0.612 106.10%
q50 39.74 38.89 -0.849 97.86%
q51 14.40 14.43 0.029 100.20%
q52 1.21 1.28 0.066 105.40%
q53 3.04 3.22 0.175 105.75%
q54 6.87 7.08 0.211 103.07%
q55 1.29 1.29 -0.003 99.73%
q56 7.09 7.96 0.862 112.15%
q57 13.54 12.62 -0.928 93.15%
q58 3.57 5.24 1.671 146.86%
q59 6.88 6.90 0.015 100.22%
q60 9.93 8.52 -1.413 85.77%
q61 9.52 8.96 -0.562 94.09%
q62 4.92 5.39 0.475 109.65%
q63 3.07 3.04 -0.026 99.14%
q64 64.02 60.07 -3.948 93.83%
q65 29.16 29.92 0.766 102.63%
q66 4.69 4.47 -0.223 95.24%
q67 226.49 223.33 -3.155 98.61%
q68 4.29 4.34 0.051 101.19%
q69 6.87 7.06 0.197 102.86%
q70 12.45 13.09 0.642 105.15%
q71 7.84 4.74 -3.105 60.40%
q72 38.78 39.69 0.910 102.35%
q73 3.98 3.39 -0.589 85.19%
q74 28.06 26.86 -1.200 95.72%
q75 43.75 44.74 0.988 102.26%
q76 14.33 14.11 -0.225 98.43%
q77 3.85 3.37 -0.481 87.50%
q78 84.03 83.87 -0.160 99.81%
q79 5.02 4.95 -0.078 98.45%
q80 17.30 16.71 -0.597 96.55%
q81 8.28 9.91 1.635 119.75%
q82 9.87 10.34 0.466 104.72%
q83 2.53 2.62 0.087 103.43%
q84 3.75 3.77 0.017 100.45%
q85 9.94 10.11 0.170 101.71%
q86 4.88 4.52 -0.364 92.55%
q87 19.20 18.60 -0.607 96.84%
q88 22.97 23.40 0.431 101.88%
q89 4.31 4.32 0.000 100.01%
q90 3.27 3.89 0.620 118.96%
q91 5.21 5.43 0.214 104.11%
q92 2.57 2.20 -0.365 85.79%
q93 54.61 54.10 -0.509 99.07%
q94 17.24 17.95 0.719 104.17%
q9 99.65 98.14 -1.515 98.48%
q5 2.97 3.19 0.220 107.40%
q96 28.18 28.38 0.196 100.70%
q97 3.43 2.68 -0.755 78.01%
q98 10.67 10.05 -0.618 94.21%
q99 10.67 10.05 -0.618 94.21%
total 2200.01 2209.01 8.997 100.41%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_01_13_2025_time.csv log/native_master_01_11_2025_a7301e199d_time.csv difference percentage
q1 44.74 42.91 -1.833 95.90%
q2 44.44 42.71 -1.738 96.09%
q3 92.10 91.84 -0.255 99.72%
q4 70.11 69.15 -0.955 98.64%
q5 178.21 179.28 1.064 100.60%
q6 18.88 17.87 -1.007 94.67%
q7 104.20 104.89 0.694 100.67%
q8 183.36 184.18 0.818 100.45%
q9 281.19 278.59 -2.599 99.08%
q10 101.09 104.92 3.828 103.79%
q11 34.24 33.65 -0.583 98.30%
q12 42.20 44.25 2.050 104.86%
q13 75.08 76.86 1.778 102.37%
q14 35.64 36.27 0.628 101.76%
q15 65.16 67.08 1.921 102.95%
q16 27.01 27.02 0.015 100.06%
q17 231.50 234.23 2.726 101.18%
q18 357.90 362.13 4.223 101.18%
q19 36.49 37.80 1.314 103.60%
q20 59.10 61.51 2.411 104.08%
q21 537.93 533.06 -4.865 99.10%
q22 25.01 24.73 -0.282 98.87%
total 2645.58 2654.94 9.353 100.35%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[VL] Scalar function name not registered: format_datetime, called with arguments: (TIMESTAMP, VARCHAR)..

3 participants