Skip to content

[SPARK-37613][SQL] Support ANSI Aggregate Function: regr_count#34880

Closed
beliefer wants to merge 15 commits intoapache:masterfrom
beliefer:SPARK-37613
Closed

[SPARK-37613][SQL] Support ANSI Aggregate Function: regr_count#34880
beliefer wants to merge 15 commits intoapache:masterfrom
beliefer:SPARK-37613

Conversation

@beliefer
Copy link
Contributor

@beliefer beliefer commented Dec 13, 2021 •
edited
Loading

What changes were proposed in this pull request?

This PR used to support ANSI aggregate Function: regr_count

The mainstream database supports regr_count show below:
Teradata
https://docs.teradata.com/r/kmuOwjp1zEYg98JsB8fu_A/1skTm5SD0HD3Z~vii3hXAg
Snowflake
https://docs.snowflake.com/en/sql-reference/functions/regr_count.html#
Oracle
https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/REGR_-Linear-Regression-Functions.html#GUID-A675B68F-2A88-4843-BE2C-FCDE9C65F9A9
H2
http://www.h2database.com/html/functions-aggregate.html#regr_count
Postgresql
https://www.postgresql.org/docs/8.4/functions-aggregate.html
Sybase
https://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.sqlanywhere.12.0.0/dbreference/regr-count-function.html
Exasol
https://docs.exasol.com/sql_references/functions/alphabeticallistfunctions/regr_function.htm

Why are the changes needed?

regr_count is very useful.

Does this PR introduce any user-facing change?

'No'. New feature.

How was this patch tested?

New tests.

@github-actions github-actions bot added the SQL label Dec 13, 2021
@SparkQA
Copy link

SparkQA commented Dec 13, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50610/

@SparkQA
Copy link

SparkQA commented Dec 13, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50610/

@SparkQA
Copy link

SparkQA commented Dec 13, 2021

Test build #146136 has finished for PR 34880 at commit 268cc7b.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50626/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50627/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50626/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50627/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Test build #146154 has finished for PR 34880 at commit afe9695.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

beliefer commented Dec 14, 2021

ping @MaxGekk @gengliangwang cc @cloud-fan

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50646/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Test build #146153 has finished for PR 34880 at commit c8bd0ca.

  • This patch fails from timeout after a configured wait of 500m.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • case class RegrCount(left: Expression, right: Expression)

@beliefer
Copy link
Contributor Author

beliefer commented Dec 14, 2021

retest this please

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50646/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50655/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50655/

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Test build #146173 has finished for PR 34880 at commit f3651d9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 14, 2021

Test build #146182 has finished for PR 34880 at commit f3651d9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Test build #146248 has finished for PR 34880 at commit 4e026be.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

beliefer commented Dec 16, 2021

retest this please

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50736/

@cloud-fan
Copy link
Contributor

cloud-fan commented Dec 16, 2021

please also address #34880 (comment)

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50739/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50736/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50739/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50744/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50749/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50744/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Test build #146269 has finished for PR 34880 at commit 9dfbae4.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50749/

@beliefer
Copy link
Contributor Author

beliefer commented Dec 16, 2021

retest this please

@cloud-fan
Copy link
Contributor

cloud-fan commented Dec 16, 2021

please also rerun the github actions (I don't have permissions to do so as the github action runs on your account)

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Test build #146264 has finished for PR 34880 at commit 4e026be.

  • This patch fails SparkR unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@beliefer
Copy link
Contributor Author

beliefer commented Dec 16, 2021

please also rerun the github actions (I don't have permissions to do so as the github action runs on your account)

OK

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50757/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50757/

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Test build #146274 has finished for PR 34880 at commit d0e6bb7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 16, 2021

Test build #146283 has finished for PR 34880 at commit d0e6bb7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor

cloud-fan commented Dec 17, 2021

thanks, merging to master!

@cloud-fan cloud-fan closed this in e1c8cc1 Dec 17, 2021
@beliefer
Copy link
Contributor Author

beliefer commented Dec 17, 2021

@cloud-fan Thanks a lot!

@beliefer beliefer mentioned this pull request Dec 20, 2021
Closed
@beliefer beliefer mentioned this pull request Apr 19, 2022
Closed
HyukjinKwon pushed a commit that referenced this pull request Apr 20, 2022
@beliefer @HyukjinKwon
[SPARK-37613][SQL][FOLLOWUP] Supplement docs for regr_count
1b106ea 
### What changes were proposed in this pull request?
#34880 supported ANSI Aggregate Function: regr_count.
But the docs of regr_count is not good enough.

### Why are the changes needed?
Make the docs of regr_count more detailed.

### Does this PR introduce _any_ user-facing change?
'No'.
New feature.

### How was this patch tested?
N/A

Closes #36258 from beliefer/SPARK-37613_followup.

Authored-by: Jiaan Geng <beliefer@163.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
HyukjinKwon pushed a commit that referenced this pull request Apr 20, 2022
@beliefer @HyukjinKwon
[SPARK-37613][SQL][FOLLOWUP] Supplement docs for regr_count
2b3df38 
### What changes were proposed in this pull request?
#34880 supported ANSI Aggregate Function: regr_count.
But the docs of regr_count is not good enough.

### Why are the changes needed?
Make the docs of regr_count more detailed.

### Does this PR introduce _any_ user-facing change?
'No'.
New feature.

### How was this patch tested?
N/A

Closes #36258 from beliefer/SPARK-37613_followup.

Authored-by: Jiaan Geng <beliefer@163.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit 1b106ea)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cloud-fan cloud-fan cloud-fan approved these changes

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@beliefer @SparkQA @cloud-fan