[SPARK-31383][SQL][DOC] Clean up the SQL documents in docs/sql-ref* #28151

maropu · 2020-04-08T08:00:12Z

What changes were proposed in this pull request?

This PR intends to clean up the SQL documents in doc/sql-ref*.
Main changes are as follows;

Fixes wrong syntaxes and capitalize sub-titles
Adds some DDL queries in Examples so that users can run examples there
Makes query output in Examples follows the Dataset.showString (right-aligned) format
Adds/Removes spaces, Indents, or blank lines to follow the format below;

---
license...
---

### Description

Writes what's the syntax is.

### Syntax

{% highlight sql %}
SELECT...
    WHERE... // 4 indents after the second line
    ...
{% endhighlight %}

### Parameters

<dl>
  <!-- 2 indents in this section -->
  <dt><code><em>Param Name</em></code></dt>
  <dd>
    Param Description
  </dd>
  ...
</dl>

### Examples

{% highlight sql %}
-- It is better that users are able to execute example queries here.
-- So, we prepare test data in the first section if possible.
CREATE TABLE t (key STRING, value DOUBLE);
INSERT INTO t VALUES
    ('a', 1.0), ('a', 2.0), ('b', 3.0), ('c', 4.0);

-- query output has 2 indents and it follows the `Dataset.showString`
-- format (right-aligned).
SELECT * FROM t;
  +---+-----+
  |key|value|
  +---+-----+
  |  a|  1.0|
  |  a|  2.0|
  |  b|  3.0|
  |  c|  4.0|
  +---+-----+

-- Query statements after the second line have 4 indents.
SELECT key, SUM(value)
    FROM t
    GROUP BY key;
  +---+----------+                                                                
  |key|sum(value)|
  +---+----------+
  |  c|       4.0|
  |  b|       3.0|
  |  a|       3.0|
  +---+----------+
...
{% endhighlight %}

### Related Statements

 * [XXX](xxx.html)
 * ...

Why are the changes needed?

The most changes of this PR are pretty minor, but I think the consistent formats/rules to write documents are important for long-term maintenance in our community

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Manually checked.

maropu · 2020-04-08T08:07:18Z

@gatorsmile @huaxingao @kevinyu98 @srowen I've roughly checked the current whole SQL docs and fixed minor&trivial issues for consistency across the docs. How about this update?

SparkQA · 2020-04-08T08:13:00Z

Test build #120959 has finished for PR 28151 at commit 0c56949.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

If it's just cleanup, fine. This is for 3.0?
I skimmed the changes and it looks indeed like just formatting, etc.

huaxingao · 2020-04-08T15:49:25Z

docs/sql-ref-null-semantics.md

-  |Fred|50 |
-  |Dan |50 |
+  |Fred| 50|
+  | Da | 50|


huaxingao · 2020-04-08T16:00:27Z

docs/sql-ref-syntax-aux-analyze-table.md

+  |            col_name|           data_type|comment|
+  +--------------------+--------------------+-------+
+  |                name|              string|   null|
+  |          student_id|                 int|   null|


null -> NULL?
Shall we change all the null to NULL in tables?

Ur, null looks ok because Dataset.showString does so;

scala> Seq(Some(1), None).toDF("a").show() +----+ | a| +----+ | 1| |null| +----+

These are SQL examples. SQL uses NULL?

I'm not faimilar with that though, the output format looks implementation-specific. For example, PostgreSQL uses an empty cell for null by default. Anyway, I like the simple document rule like Query output in Examples should follow the Dataset.showString (right-aligned) format. Since we can easily replace it later, I leave it as it is in this PR. If necessary, please do follow-up later.

huaxingao · 2020-04-08T16:00:46Z

docs/sql-ref-syntax-ddl-truncate-table.md

+  |name|rollno|age|
+  +----+------+---+
+  +----+------+---+
+  No rows selected 


nit: This line is a comment, right? put -- in front of it?

Ah, yes. Probably, we can remove this. Thanks!

huaxingao · 2020-04-08T16:10:03Z

@maropu Thanks a lot for doing this!

huaxingao · 2020-04-08T16:11:16Z

This is for 3.0. @srowen

maropu · 2020-04-08T23:49:05Z

Yea, this is just a clean-up for 3.0.

maropu · 2020-04-08T23:55:34Z

docs/sql-ref-syntax-qry-select-setops.md

+
+#### Examples
+
+{% highlight sql %}


@huaxingao I moved the examples into the seb-sections of EXCEPT/INTERSECT/UNION for readability. Could you check this?

Looks good. Thanks!

maropu · 2020-04-08T23:57:55Z

@huaxingao @kevinyu98 If no problem, could you apply the same cleanup for your open PRs at your convenience?

SparkQA · 2020-04-09T00:03:14Z

Test build #120988 has finished for PR 28151 at commit a7c5e23.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

huaxingao · 2020-04-09T00:06:20Z

I will reformat my open PRs.

kevinyu98 · 2020-04-09T05:13:10Z

docs/sql-ref-syntax-aux-show-table.md

+                                   |-- name: string (nullable = true)
+                                   |-- grade: integer (nullable = true)
+                                                                                                            
+  +--------+---------+-----------+--------------------------------------------------------------+


does this output need to be right-aligned? also the following SHOW TABLE output.

Yea right, but the information cell itself assumes to be left-aligned, so I left it as it is. Probably, this is an exception case.

kevinyu98 · 2020-04-09T05:35:05Z

docs/sql-ref-syntax-ddl-alter-table.md

+  |   age=11|
+  |   age=12|
+  |   age=15|
+  +---------+

 ALTER TABLE StudentInfo ADD IF NOT EXISTS PARTITION (age=18);

 -- After adding a new partition to the table


Do we need to leave space after --? I saw some have one space, some didn't have.

yea, you're right. I'll remove that. thanks.

SparkQA · 2020-04-09T07:05:02Z

Test build #121008 has finished for PR 28151 at commit 5f6677a.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

maropu · 2020-04-09T07:08:36Z

retest this please

SparkQA · 2020-04-09T07:26:03Z

Test build #121012 has finished for PR 28151 at commit 5f6677a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

kevinyu98 · 2020-04-09T16:51:27Z

docs/sql-ref-syntax-qry-explain.md


--Default Output
+{% highlight sql %}
+-- Default Output



do we need leave space here? also the following -- in this file.

SparkQA · 2020-04-10T04:40:17Z

Test build #121058 has finished for PR 28151 at commit 945b28c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

maropu · 2020-04-10T08:14:19Z

@srowen Could you check?

huaxingao · 2020-04-10T17:22:53Z

LGTM

SparkQA · 2020-04-13T00:59:51Z

Test build #121156 has finished for PR 28151 at commit 658fc77.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-04-13T01:44:12Z

Test build #121158 has finished for PR 28151 at commit 2022c42.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

### What changes were proposed in this pull request? This PR intends to clean up the SQL documents in `doc/sql-ref*`. Main changes are as follows; - Fixes wrong syntaxes and capitalize sub-titles - Adds some DDL queries in `Examples` so that users can run examples there - Makes query output in `Examples` follows the `Dataset.showString` (right-aligned) format - Adds/Removes spaces, Indents, or blank lines to follow the format below; ``` --- license... --- ### Description Writes what's the syntax is. ### Syntax {% highlight sql %} SELECT... WHERE... // 4 indents after the second line ... {% endhighlight %} ### Parameters <dl> <dt><code><em>Param Name</em></code></dt> <dd> Param Description </dd> ... </dl> ### Examples {% highlight sql %} -- It is better that users are able to execute example queries here. -- So, we prepare test data in the first section if possible. CREATE TABLE t (key STRING, value DOUBLE); INSERT INTO t VALUES ('a', 1.0), ('a', 2.0), ('b', 3.0), ('c', 4.0); -- query output has 2 indents and it follows the `Dataset.showString` -- format (right-aligned). SELECT * FROM t; +---+-----+ |key|value| +---+-----+ | a| 1.0| | a| 2.0| | b| 3.0| | c| 4.0| +---+-----+ -- Query statements after the second line have 4 indents. SELECT key, SUM(value) FROM t GROUP BY key; +---+----------+ |key|sum(value)| +---+----------+ | c| 4.0| | b| 3.0| | a| 3.0| +---+----------+ ... {% endhighlight %} ### Related Statements * [XXX](xxx.html) * ... ``` ### Why are the changes needed? The most changes of this PR are pretty minor, but I think the consistent formats/rules to write documents are important for long-term maintenance in our community ### Does this PR introduce any user-facing change? Yes. ### How was this patch tested? Manually checked. Closes #28151 from maropu/MakeRightAligned. Authored-by: Takeshi Yamamuro <yamamuro@apache.org> Signed-off-by: Sean Owen <srowen@gmail.com> (cherry picked from commit 179289f) Signed-off-by: Sean Owen <srowen@gmail.com>

srowen · 2020-04-13T04:40:52Z

Merged to master/3.0

### What changes were proposed in this pull request? This PR intends to clean up the SQL documents in `doc/sql-ref*`. Main changes are as follows; - Fixes wrong syntaxes and capitalize sub-titles - Adds some DDL queries in `Examples` so that users can run examples there - Makes query output in `Examples` follows the `Dataset.showString` (right-aligned) format - Adds/Removes spaces, Indents, or blank lines to follow the format below; ``` --- license... --- ### Description Writes what's the syntax is. ### Syntax {% highlight sql %} SELECT... WHERE... // 4 indents after the second line ... {% endhighlight %} ### Parameters <dl> <dt><code><em>Param Name</em></code></dt> <dd> Param Description </dd> ... </dl> ### Examples {% highlight sql %} -- It is better that users are able to execute example queries here. -- So, we prepare test data in the first section if possible. CREATE TABLE t (key STRING, value DOUBLE); INSERT INTO t VALUES ('a', 1.0), ('a', 2.0), ('b', 3.0), ('c', 4.0); -- query output has 2 indents and it follows the `Dataset.showString` -- format (right-aligned). SELECT * FROM t; +---+-----+ |key|value| +---+-----+ | a| 1.0| | a| 2.0| | b| 3.0| | c| 4.0| +---+-----+ -- Query statements after the second line have 4 indents. SELECT key, SUM(value) FROM t GROUP BY key; +---+----------+ |key|sum(value)| +---+----------+ | c| 4.0| | b| 3.0| | a| 3.0| +---+----------+ ... {% endhighlight %} ### Related Statements * [XXX](xxx.html) * ... ``` ### Why are the changes needed? The most changes of this PR are pretty minor, but I think the consistent formats/rules to write documents are important for long-term maintenance in our community ### Does this PR introduce any user-facing change? Yes. ### How was this patch tested? Manually checked. Closes apache#28151 from maropu/MakeRightAligned. Authored-by: Takeshi Yamamuro <yamamuro@apache.org> Signed-off-by: Sean Owen <srowen@gmail.com>

srowen reviewed Apr 8, 2020

View reviewed changes

huaxingao reviewed Apr 8, 2020

View reviewed changes

maropu force-pushed the MakeRightAligned branch from 0c56949 to 1abe5b8 Compare April 8, 2020 23:46

maropu commented Apr 8, 2020

View reviewed changes

maropu mentioned this pull request Apr 9, 2020

[SPARK-31349][SQL][DOCS] Document built-in aggregate functions in SQL Reference #28120

Closed

kevinyu98 reviewed Apr 9, 2020

View reviewed changes

maropu force-pushed the MakeRightAligned branch from 5f6677a to 945b28c Compare April 10, 2020 04:23

maropu added 2 commits April 13, 2020 09:40

Fix

42ccdcb

Fix

658fc77

maropu force-pushed the MakeRightAligned branch from 945b28c to 658fc77 Compare April 13, 2020 00:41

Fix

2022c42

probot-autolabeler bot added the DOCS label Apr 13, 2020

srowen closed this in 179289f Apr 13, 2020

maropu mentioned this pull request Apr 20, 2020

[SPARK-31331][SQL][DOCS] Document Spark integration with Hive UDFs/UDAFs/UDTFs #28104

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-31383][SQL][DOC] Clean up the SQL documents in docs/sql-ref* #28151

[SPARK-31383][SQL][DOC] Clean up the SQL documents in docs/sql-ref* #28151

maropu commented Apr 8, 2020

maropu commented Apr 8, 2020

SparkQA commented Apr 8, 2020

srowen left a comment

huaxingao Apr 8, 2020

huaxingao Apr 8, 2020

maropu Apr 8, 2020

huaxingao Apr 9, 2020

maropu Apr 9, 2020

huaxingao Apr 9, 2020

huaxingao Apr 8, 2020

maropu Apr 8, 2020

huaxingao commented Apr 8, 2020

huaxingao commented Apr 8, 2020

maropu commented Apr 8, 2020

maropu Apr 8, 2020

huaxingao Apr 9, 2020

maropu commented Apr 8, 2020

SparkQA commented Apr 9, 2020

huaxingao commented Apr 9, 2020

kevinyu98 Apr 9, 2020

maropu Apr 9, 2020

kevinyu98 Apr 9, 2020

kevinyu98 Apr 9, 2020

maropu Apr 9, 2020

SparkQA commented Apr 9, 2020

maropu commented Apr 9, 2020

SparkQA commented Apr 9, 2020

kevinyu98 Apr 9, 2020

SparkQA commented Apr 10, 2020

maropu commented Apr 10, 2020

huaxingao commented Apr 10, 2020

SparkQA commented Apr 13, 2020

SparkQA commented Apr 13, 2020

srowen commented Apr 13, 2020

[SPARK-31383][SQL][DOC] Clean up the SQL documents in docs/sql-ref* #28151

[SPARK-31383][SQL][DOC] Clean up the SQL documents in docs/sql-ref* #28151

Conversation

maropu commented Apr 8, 2020

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

maropu commented Apr 8, 2020

SparkQA commented Apr 8, 2020

srowen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

huaxingao commented Apr 8, 2020

huaxingao commented Apr 8, 2020

maropu commented Apr 8, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maropu commented Apr 8, 2020

SparkQA commented Apr 9, 2020

huaxingao commented Apr 9, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Apr 9, 2020

maropu commented Apr 9, 2020

SparkQA commented Apr 9, 2020

Choose a reason for hiding this comment

SparkQA commented Apr 10, 2020

maropu commented Apr 10, 2020

huaxingao commented Apr 10, 2020

SparkQA commented Apr 13, 2020

SparkQA commented Apr 13, 2020

srowen commented Apr 13, 2020