update window functions doc #15902

techdocsmith · 2024-02-13T22:41:32Z

Adds more detail about window frames.
Adds information about the strict window frame check: #15746

This PR has:

[x ] been self-reviewed.
- using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
added documentation for new or modified features or behaviors.
a release note entry in the PR description.
added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
added or updated version, license, or notice information in licenses.yaml
added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
added integration tests.
been tested in a test Druid cluster.

…ct to the top of the list

soumyava

First pass left some comments

soumyava · 2024-02-15T16:52:16Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the cumulative distribution of the current row within the window calculated as `number of window rows at the same rank or higher than current row` / `total window rows`.


Should we indicate if the value ranges between 1/#rows and 1 similar to what postgres does ?

soumyava · 2024-02-15T16:53:23Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the value for the expression for the first row within the window.


nit. Maybe value evaluated for the expression

soumyava · 2024-02-15T16:55:14Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the value evaluated at the row that precedes the current row by the offset number within the window. `offset` defaults to 1 if not provided.


Maybe make it a bit clear like returns the value evaluated at the row which is offset rows preceding the current row within the window

soumyava · 2024-02-15T16:56:14Z

docs/querying/sql-functions.md

@@ -876,6 +916,14 @@ Returns the value of a numeric or string expression corresponding to the latest

 Returns the value of a numeric or string expression corresponding to the latest time value from `timestampExpr`.

+## LEAD


Is this sorted alphabetically, since LEAD and LAG are similar I expected them to be together but if alphabetical it's all good too

alphabetically. window functions are mixed in with others.

soumyava · 2024-02-15T16:56:33Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the value evaluated at the row that follows the current row by the offset number within the window; if there is no such row, returns the given default value. `offset` defaults to 1 if not provided.


Maybe similar wording as lag

soumyava · 2024-02-15T17:02:47Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the rank of the row calculated as a percentage according to the formula: `(rank - 1) / (total window rows - 1)`.


nit. Returns relative rank of the current row

probably an alternate way to write the formula would be RANK() OVER (window) / COUNT(1) OVER (window)

soumyava · 2024-02-15T17:03:29Z

docs/querying/sql-functions.md

+
+`RANK()`
+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)


Should specify with gaps. @kgyrtkirk please chime in

soumyava · 2024-02-15T17:04:40Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the number of the row within the window.


We should mention counting from 1 as postgres does.

soumyava · 2024-02-15T17:05:35Z

docs/querying/sql-window-functions.md

+    dimensions,
+    aggregation function(s)
+    window_function()
+      OVER ( PARTITION BY partitioning expression


[PARTITION BY ...] [ORDER BY ...] as these can be optional

I think the section about window frames could be moved up here; as that gives a better understanding of partition by and other parts of the frame.

As this sections is about syntax - possibly remove the select and other non-related things; and only keep

window_function() OVER window

I think it would be important to somehow show that the window can be specified later as well

window_function() OVER w from t WINDOW w AS (PARTITION BY ... )

soumyava · 2024-02-15T17:06:16Z

docs/querying/sql-window-functions.md

+...
+```
+
+Druid applies the GROUP BY dimensions first before calculating all non-window aggregation functions. Then it applies the window function over the aggregate results.


Should also specify what an empty OVER() indicates

@soumyava, @kgyrtkirk what does an empty OVER() indicate?

empty OVER() or absence of a partition by indicates that all the data belong to a single window

soumyava · 2024-02-15T17:44:40Z

docs/querying/sql-window-functions.md

+ORDER BY channel,TIME_FLOOR(__time, 'PT1H'), user
+```
+
+The windows only define the PARTITION BY clause of the window, so Druid performs the calculation over the whole dataset for each value of the partition expression.


This seems to repeat with the earlier line 150

kgyrtkirk · 2024-02-16T12:41:04Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Divides the rows within a window as evenly as possible into the number of tiles, also called buckets, and returns the value of the tile that the row falls into.


NTILE(2) will return 1 and 2 - I think we are good

kgyrtkirk · 2024-02-16T12:48:43Z

docs/querying/sql-functions.md

+
+**Function type:** [Window](sql-window-functions.md#window-function-reference)
+
+Returns the rank of the row calculated as a percentage according to the formula: `(rank - 1) / (total window rows - 1)`.


probably an alternate way to write the formula would be RANK() OVER (window) / COUNT(1) OVER (window)

kgyrtkirk · 2024-02-16T12:59:32Z

docs/querying/sql-window-functions.md

+```sql
+window function
+OVER (
+        [ PARTITION BY partition expression] ORDER BY order expression


ORDER BY is not mandatory

kgyrtkirk · 2024-02-16T13:10:42Z

docs/querying/sql-window-functions.md

+    dimensions,
+    aggregation function(s)
+    window_function()
+      OVER ( PARTITION BY partitioning expression


I think the section about window frames could be moved up here; as that gives a better understanding of partition by and other parts of the frame.

As this sections is about syntax - possibly remove the select and other non-related things; and only keep

window_function() OVER window

I think it would be important to somehow show that the window can be specified later as well

window_function() OVER w from t WINDOW w AS (PARTITION BY ... )

soumyava · 2024-02-21T19:06:51Z

Apart from adding a line about the empty OVER() rest LGTM !

docs/querying/sql-window-functions.md

soumyava

LGTM !

docs/querying/sql-functions.md

docs/querying/sql-window-functions.md

soumyava

Need a spelling change to get the static check to pass

docs/querying/sql-functions.md

docs/querying/sql-window-functions.md

docs/querying/sql-functions.md

docs/querying/sql-window-functions.md

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

docs/querying/sql-window-functions.md

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

docs/querying/sql-window-functions.md

vtlim · 2024-03-07T22:07:20Z

docs/querying/sql-window-functions.md

+- _N_ ROWS PRECEDING: _N_ rows before the current row as ordered by the order expression
+- CURRENT ROW: the current row
+- _N_ ROWS FOLLOWING: _N_ rows after the current row as ordered by the order expression
+- _UNBOUNDED FOLLOWING_: to the end of the window as ordered by the order expression


Suggested change

- _UNBOUNDED FOLLOWING_: to the end of the window as ordered by the order expression

- UNBOUNDED FOLLOWING: to the end of the window as ordered by the order expression

Note that this one says UNBOUNDED but line 131 says UNBOUND. Should they be the same or is this intentional?

docs/querying/sql-window-functions.md

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

vtlim

The updates LGTM 🦖

techdocsmith added 2 commits February 13, 2024 14:16

update window functions doc

78fe675

add window functions to the all functions reference. move the referen…

b5b9679

…ct to the top of the list

github-actions bot added the Area - Documentation label Feb 13, 2024

techdocsmith added 2 commits February 13, 2024 14:56

fix spelling

b3b1c40

fix spelling

dcbda4d

techdocsmith requested review from 317brian and soumyava February 13, 2024 23:30

soumyava reviewed Feb 15, 2024

View reviewed changes

kgyrtkirk reviewed Feb 16, 2024

View reviewed changes

address comments

ae99280

techdocsmith commented Feb 21, 2024

View reviewed changes

docs/querying/sql-window-functions.md Show resolved Hide resolved

Update docs/querying/sql-window-functions.md

29421a8

soumyava approved these changes Feb 21, 2024

View reviewed changes

techdocsmith commented Feb 21, 2024

View reviewed changes

docs/querying/sql-functions.md Outdated Show resolved Hide resolved

techdocsmith commented Feb 21, 2024

View reviewed changes

docs/querying/sql-window-functions.md Outdated Show resolved Hide resolved

fix spelling

6b17df3

soumyava reviewed Feb 22, 2024

View reviewed changes

docs/querying/sql-functions.md Outdated Show resolved Hide resolved

docs/querying/sql-window-functions.md Outdated Show resolved Hide resolved

vtlim reviewed Feb 23, 2024

View reviewed changes

techdocsmith and others added 2 commits February 26, 2024 14:05

Apply suggestions from code review

bb4d4a6

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

Apply suggestions from code review

5f33727

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

techdocsmith commented Feb 26, 2024

View reviewed changes

docs/querying/sql-window-functions.md Outdated Show resolved Hide resolved

techdocsmith and others added 3 commits February 26, 2024 14:13

Apply suggestions from code review

77dc1b2

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

address review comments

763b237

include results for sample queries

abc4e9f

vtlim reviewed Mar 7, 2024

View reviewed changes

docs/querying/sql-window-functions.md Outdated Show resolved Hide resolved

techdocsmith and others added 2 commits March 7, 2024 14:28

Update docs/querying/sql-window-functions.md

3102b71

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

Update docs/querying/sql-window-functions.md

4a56e65

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

techdocsmith and others added 2 commits March 7, 2024 14:37

Apply suggestions from code review

6c49168

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

format table results

59234cb

vtlim approved these changes Mar 7, 2024

View reviewed changes

vtlim merged commit 3caacba into apache:master Mar 7, 2024
11 checks passed

adarshsanjeev added this to the 30.0.0 milestone May 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update window functions doc #15902

update window functions doc #15902

techdocsmith commented Feb 13, 2024

soumyava left a comment

soumyava Feb 15, 2024

soumyava Feb 15, 2024

soumyava Feb 15, 2024

soumyava Feb 15, 2024

techdocsmith Feb 21, 2024

soumyava Feb 15, 2024

soumyava Feb 15, 2024

kgyrtkirk Feb 16, 2024

soumyava Feb 15, 2024

soumyava Feb 15, 2024

soumyava Feb 15, 2024

kgyrtkirk Feb 16, 2024

soumyava Feb 15, 2024

techdocsmith Feb 21, 2024

soumyava Feb 21, 2024

soumyava Feb 15, 2024

kgyrtkirk Feb 16, 2024

kgyrtkirk Feb 16, 2024

kgyrtkirk Feb 16, 2024

kgyrtkirk Feb 16, 2024

soumyava commented Feb 21, 2024

soumyava left a comment

soumyava left a comment

vtlim Mar 7, 2024

vtlim Mar 7, 2024

vtlim left a comment


		Function type: [Window](sql-window-functions.md#window-function-reference)

		Returns the cumulative distribution of the current row within the window calculated as `number of window rows at the same rank or higher than current row` / `total window rows`.


		Function type: [Window](sql-window-functions.md#window-function-reference)

		Returns the value for the expression for the first row within the window.


		Function type: [Window](sql-window-functions.md#window-function-reference)

		Returns the value evaluated at the row that precedes the current row by the offset number within the window. `offset` defaults to 1 if not provided.

		@@ -876,6 +916,14 @@ Returns the value of a numeric or string expression corresponding to the latest

		Returns the value of a numeric or string expression corresponding to the latest time value from `timestampExpr`.

		## LEAD


		Function type: [Window](sql-window-functions.md#window-function-reference)

		Returns the rank of the row calculated as a percentage according to the formula: `(rank - 1) / (total window rows - 1)`.


		`RANK()`

		Function type: [Window](sql-window-functions.md#window-function-reference)


		Function type: [Window](sql-window-functions.md#window-function-reference)

		Returns the number of the row within the window.


		Function type: [Window](sql-window-functions.md#window-function-reference)

		Divides the rows within a window as evenly as possible into the number of tiles, also called buckets, and returns the value of the tile that the row falls into.

	- _UNBOUNDED FOLLOWING_: to the end of the window as ordered by the order expression
	- UNBOUNDED FOLLOWING: to the end of the window as ordered by the order expression

update window functions doc #15902

update window functions doc #15902

Conversation

techdocsmith commented Feb 13, 2024

soumyava left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soumyava commented Feb 21, 2024

soumyava left a comment

Choose a reason for hiding this comment

soumyava left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vtlim left a comment

Choose a reason for hiding this comment