[SPARK-31326][SQL][DOCS] Create Function docs structure for SQL Reference #28099
Conversation
| This guide is a reference for Structured Query Language (SQL) for Apache | ||
| Spark. This document describes the SQL constructs supported by Spark in detail | ||
| along with usage examples when applicable. | ||
| Spark SQL defines built-in functions to use, a complete list of which can be found [here](api/sql/). Among them, Spark SQL has several special categories of built-in functions: [Aggregate Functions](sql-ref-functions-builtin-aggregate.html) to operate on a group of rows, [Array Functions](sql-ref-functions-builtin-array.html) to operate on Array columns, and [Date and Time Functions](sql-ref-functions-builtin-date-time.html) to operate on Date and Time. |
There was a problem hiding this comment.
[here](api/sql/) links to the built-in function page https://spark.apache.org/docs/latest/api/sql/
There was a problem hiding this comment.
How did you choose these three special catalogries decumented here?
There was a problem hiding this comment.
I initially wanted to have scalar built-in functions and aggregate built-in functions to be parallel to the sections in UDFs, but then I think it's not worthwhile to document scalar built-in functions separately. I guess it makes more sense to document some special functions, such as array functions, date and time functions, and maybe window functions and string functions as well. I also don't want to document too many here, because I need to finish quickly.
There was a problem hiding this comment.
You mean that just the link to the full bulitin method list is not enough? What are you planning to describe here about the special three categories? more examples?
There was a problem hiding this comment.
I saw other SQL references (such as Oracle, Hive, Snowflake) document the built-in functions in different categories. I think it's nice for us to have that too. The full built-in function list doesn't separate the functions into different categories. I think we at least need to document the aggregate functions.
|
|
||
| * [Scalar User-Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html) | ||
| * [User-Defined Aggregate Functions (UDAFs)](sql-ref-functions-udf-aggregate.html) | ||
| * [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html) |
There was a problem hiding this comment.
I will add a page for Hive UDFs
|
Test build #120715 has finished for PR 28099 at commit
|
…ence ### What changes were proposed in this pull request? Create Function docs structure for SQL Reference... ### Why are the changes needed? so the Function docs can be added later, also want to get a consensus about what to document for Functions in SQL Reference. ### Does this PR introduce any user-facing change? Yes <img width="1050" alt="Screen Shot 2020-04-02 at 12 09 20 AM" src="https://user-images.githubusercontent.com/13592258/78220451-68b6e100-7476-11ea-9a21-733b41652785.png"> <img width="1051" alt="Screen Shot 2020-04-02 at 12 09 44 AM" src="https://user-images.githubusercontent.com/13592258/78220460-6ce2fe80-7476-11ea-887c-defefd55c19d.png"> <img width="1051" alt="Screen Shot 2020-04-02 at 12 10 05 AM" src="https://user-images.githubusercontent.com/13592258/78220463-6f455880-7476-11ea-81fc-fd4137db7c3f.png"> ### How was this patch tested? Manually build and check Closes #28099 from huaxingao/function. Authored-by: Huaxin Gao <huaxing@us.ibm.com> Signed-off-by: Takeshi Yamamuro <yamamuro@apache.org> (cherry picked from commit 4e45c07) Signed-off-by: Takeshi Yamamuro <yamamuro@apache.org>
|
Thanks! Merged to master/3.0 |
|
Thanks you all! |
…ence ### What changes were proposed in this pull request? Create Function docs structure for SQL Reference... ### Why are the changes needed? so the Function docs can be added later, also want to get a consensus about what to document for Functions in SQL Reference. ### Does this PR introduce any user-facing change? Yes <img width="1050" alt="Screen Shot 2020-04-02 at 12 09 20 AM" src="https://user-images.githubusercontent.com/13592258/78220451-68b6e100-7476-11ea-9a21-733b41652785.png"> <img width="1051" alt="Screen Shot 2020-04-02 at 12 09 44 AM" src="https://user-images.githubusercontent.com/13592258/78220460-6ce2fe80-7476-11ea-887c-defefd55c19d.png"> <img width="1051" alt="Screen Shot 2020-04-02 at 12 10 05 AM" src="https://user-images.githubusercontent.com/13592258/78220463-6f455880-7476-11ea-81fc-fd4137db7c3f.png"> ### How was this patch tested? Manually build and check Closes apache#28099 from huaxingao/function. Authored-by: Huaxin Gao <huaxing@us.ibm.com> Signed-off-by: Takeshi Yamamuro <yamamuro@apache.org>
What changes were proposed in this pull request?
Create Function docs structure for SQL Reference...
Why are the changes needed?
so the Function docs can be added later, also want to get a consensus about what to document for Functions in SQL Reference.
Does this PR introduce any user-facing change?
Yes
How was this patch tested?
Manually build and check