Skip to content

Conversation

@vinodkc
Copy link
Contributor

@vinodkc vinodkc commented Apr 10, 2025

This PR

  1. Modifies the EXTRACT method to support the TIME data type.

Supported values of field when source is TIME:

HOUR, H, HOURS, HR, HRS: The hour field (0 - 23)
MINUTE, M, MIN, MINS, MINUTES: The minutes field (0 - 59)
SECOND, S, SEC, SECONDS, SECS: The seconds field, including fractional parts (a DECIMAL(8, 6))

scala> spark.sql("SELECT extract(HOUR FROM time'09:08:01.987654')").show
+-----------------------------------------+
|extract(HOUR FROM TIME '09:08:01.987654')|
+-----------------------------------------+
|                                        9|
+-----------------------------------------+

scala> spark.sql("SELECT extract(MINUTE FROM time'09:08:01.987654')").show
+-------------------------------------------+
|extract(MINUTE FROM TIME '09:08:01.987654')|
+-------------------------------------------+
|                                          8|
+-------------------------------------------+

scala> spark.sql("SELECT extract(SECOND FROM time'09:08:01.987654')").show
+-------------------------------------------+
|extract(SECOND FROM TIME '09:08:01.987654')|
+-------------------------------------------+
|                                   1.987654|
+-------------------------------------------+

With TimeType precisions 0 to 6

spark.sql("""select
extract(SECOND FROM cast('09:08:01.987654' as time(0))) as t0,
extract(SECOND FROM cast('09:08:01.987654' as time(1))) as t1,
extract(SECOND FROM cast('09:08:01.987654' as time(2))) as t2,
extract(SECOND FROM cast('09:08:01.987654' as time(3))) as t3,
extract(SECOND FROM cast('09:08:01.987654' as time(4))) as t4,
extract(SECOND FROM cast('09:08:01.987654' as time(5))) as t5,
extract(SECOND FROM cast('09:08:01.987654' as time(6))) as t6
""").show
+--------+--------+--------+--------+--------+--------+--------+
|      t0|      t1|      t2|      t3|      t4|      t5|      t6|
+--------+--------+--------+--------+--------+--------+--------+
|1.000000|1.900000|1.980000|1.987000|1.987600|1.987650|1.987654|
+--------+--------+--------+--------+--------+--------+--------+

Note : secondoftime_with_fraction added to support seconds with fraction in the built-in function extract()
eg:
SELECT extract(SECOND FROM time'09:08:01.987654')

What changes were proposed in this pull request?

Why are the changes needed?

SPIP: Add the TIME data type SPARK-51162

Does this PR introduce any user-facing change?

Yes, this PR modified the existing built-in function extract to support TimeType. Also added a new built-in function secondoftime_with_fraction to get second of time with the fraction.

How was this patch tested?

Manual testing, as shown above, and UTs added:

build/sbt "sql/testOnly *ExpressionsSchemaSuite" 
build/sbt "test:testOnly *TimeExpressionsSuite.scala"
build/sbt "test:testOnly *SQLQuerySuite.scala"

Was this patch authored or co-authored using generative AI tooling?

No

@vinodkc
Copy link
Contributor Author

vinodkc commented Apr 11, 2025

@MaxGekk , Could you please review this PR?

@vinodkc vinodkc requested a review from MaxGekk April 15, 2025 01:13
@vinodkc vinodkc requested a review from MaxGekk April 18, 2025 20:43
@MaxGekk
Copy link
Member

MaxGekk commented Apr 19, 2025

+1, LGTM. Merging to master.
Thank you, @vinodkc.

@MaxGekk MaxGekk closed this in 2e16568 Apr 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants