SQL: Implement DATEADD function #47747

matriv · 2019-10-08T20:27:20Z

Implement DATEADD/TIMESTAMPADD function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/dateadd-transact-sql?view=sql-server-2017
which allows a user to add/subtract specified number of specified units
to/from a date/datetime field/expression.

Split BinaryDateTimePipe into 2 DatePart/DateTrunc Pipe classes, so that
the BinaryDateTimeOperation enum can be removed.

Closes: #47746

Implement DATEADD/TIMESTAMPADD function as per the MS-SQL spec: https://docs.microsoft.com/en-us/sql/t-sql/functions/dateadd-transact-sql?view=sql-server-2017 which allows a user to add/subtract specified number of specified units to/from a date/datetime field/expression. Split BinaryDateTimePipe into 2 DatePart/DateTrunc Pipe classes, so that the BinaryDateTimeOperation enum can be removed. Closes: elastic#47746

elasticmachine · 2019-10-08T20:27:22Z

Pinging @elastic/es-search (:Search/SQL)

matriv · 2019-10-09T11:07:38Z

Opened a separate PR for the binary functions refactoring: #47786 which should be merged first.

costin

LGTM

costin · 2019-10-10T10:24:38Z

docs/reference/sql/functions/date-time.asciidoc

@@ -248,6 +248,79 @@ include-tagged::{sql-specs}/docs/docs.csv-spec[filterNow]
 Currently, using a _precision_ greater than 3 doesn't make any difference to the output of the
 function as the maximum number of second fractional digits returned is 3 (milliseconds).

+[[sql-functions-datetime-add]]
+==== `DATE_ADD/DATE_ADD/TIMESTAMPADD/TIMESTAMP_ADD`


DATE_ADD appears twice, one probably should be DATEADD.

costin · 2019-10-10T10:24:54Z

docs/reference/sql/functions/date-time.asciidoc

+.Synopsis:
+[source, sql]
+--------------------------------------------------
+DATE_PART(


DATE_PART ?

costin · 2019-10-10T10:26:57Z

docs/reference/sql/functions/date-time.asciidoc

+*Input*:
+
+<1> string expression denoting the unit to add to the date/datetime
+<2> integer expression denoting how many times the above unit should be added/subtracted (if it's negative)


Instead of saying added/subtracted (if it's negative) which can be confusing, explain the situation a bit.
added (negative values are allowed resulting in a subtraction. Or make another sentence - subtraction is supported through the use of negative values.

costin · 2019-10-10T10:28:30Z

docs/reference/sql/functions/date-time.asciidoc

+
+.Description:
+
+Add/Subtract the given number of units to/from a date/datetime. If any of the three arguments is `null`


Again, it's DATEADD - I get it you can effectively subtract by using negative values by using Add/Subtract implies one can do subtraction of a negative value as well which is not the case.
In other words there's only addition of both positive and negative values but never a subtraction.

costin · 2019-10-10T10:32:11Z

...org/elasticsearch/xpack/sql/expression/function/scalar/whitelist/InternalSqlScriptUtils.java

@@ -235,7 +236,7 @@ public static Double asin(Number value) {
    public static Double atan(Number value) {
        return MathOperation.ATAN.apply(value);
    }
-    
+


Thought is ok to fix the formatting since it's only touching empty lines.

astefan

Looking great. I've left some minor comments.
Also, I would like to see some more complex integration tests. For example:

SELECT first_name, gender, hire_date, DATE_ADD(CASE WHEN gender = 'M' THEN CONCAT(gender, 'onths') WHEN gender = 'F' THEN 'year' ELSE 'quarter' END, 5, hire_date + INTERVAL 10 month) AS dp, YEAR(DATE_ADD(CASE WHEN gender = 'M' THEN CONCAT(gender, 'onths') WHEN gender = 'F' THEN 'year' ELSE 'quarter' END, 5, hire_date + INTERVAL 10 month)) FROM test_emp WHERE YEAR(dp) > 1990 AND first_name LIKE '%y%' ORDER BY dp DESC LIMIT 15.
SELECT languages, first_name, gender, hire_date, CAST(DATE_ADD(CASE WHEN gender = 'M' THEN CONCAT(gender, 'onths') WHEN gender = 'F' THEN NULL ELSE 'quarter' END, 5, hire_date + INTERVAL 10 month) AS DATE) AS dp FROM test_emp WHERE languages >= 3 AND first_name LIKE '%y%' ORDER BY dp ASC, languages DESC

astefan · 2019-10-09T23:37:50Z

docs/reference/sql/functions/date-time.asciidoc

@@ -248,6 +248,79 @@ include-tagged::{sql-specs}/docs/docs.csv-spec[filterNow]
 Currently, using a _precision_ greater than 3 doesn't make any difference to the output of the
 function as the maximum number of second fractional digits returned is 3 (milliseconds).

+[[sql-functions-datetime-add]]
+==== `DATE_ADD/DATE_ADD/TIMESTAMPADD/TIMESTAMP_ADD`


DATE_ADD/DATE_ADD

astefan · 2019-10-09T23:38:24Z

docs/reference/sql/functions/date-time.asciidoc

+.Synopsis:
+[source, sql]
+--------------------------------------------------
+DATE_PART(


DATE_PART?

astefan · 2019-10-09T23:39:26Z

docs/reference/sql/functions/date-time.asciidoc

+*Input*:
+
+<1> string expression denoting the unit to add to the date/datetime
+<2> integer expression denoting how many times the above unit should be added/subtracted (if it's negative)


added (if it's positive) or subtracted (it it's negative)

astefan · 2019-10-10T08:03:39Z

docs/reference/sql/functions/date-time.asciidoc

+
+*Input*:
+
+<1> string expression denoting the unit to add to the date/datetime


I would say "the time unit".

I'm changing to date/time unit.

astefan · 2019-10-10T08:05:43Z

x-pack/plugin/sql/qa/src/main/resources/datetime.csv-spec

+null    | 2004-04-30 00:00:00.000Z
+F       | 2002-05-19 00:00:00.000Z
+;
+


All the integration tests seem to use DATE_ADD variant only. Can you mix and match the functions' aliases for completeness sake, as well?

astefan · 2019-10-10T09:18:13Z

...n/java/org/elasticsearch/xpack/sql/expression/function/scalar/datetime/DateAddProcessor.java

+
+    public static final String NAME = "dtadd";
+
+    public DateAddProcessor(Processor source1, Processor source2, Processor source3, ZoneId zoneId) {


Can you name these parameters like "datePart", "timeUnit", "dateField" or similar? (like you did in the rest of the class)

astefan · 2019-10-10T10:02:55Z

...t/java/org/elasticsearch/xpack/sql/expression/function/scalar/datetime/DateAddPipeTests.java

+    @Override
+    public void testTransform() {
+        // test transforming only the properties (source, expression),
+        // skipping the children (the two parameters of the binary function) which are tested separately


Does this comment apply to this Pipe?

Copy-paste the source of all evil :(

matriv · 2019-10-10T11:55:23Z

@astefan @costin Thanks for the review, pushed the fixes.

Implement DATEADD/TIMESTAMPADD function as per the MS-SQL spec: https://docs.microsoft.com/en-us/sql/t-sql/functions/dateadd-transact-sql?view=sql-server-2017 which allows a user to add/subtract specified number of specified units to/from a date/datetime field/expression. Closes: #47746 (cherry picked from commit e624bc2)

matriv added >feature :Analytics/SQL SQL querying v8.0.0 v7.5.0 labels Oct 8, 2019

matriv added 2 commits October 8, 2019 22:42

Fix docs and doc test

eb81698

Added integration tests

2b274e6

matriv requested a review from astefan October 9, 2019 10:01

matriv assigned costin and bpintea and unassigned costin and bpintea Oct 9, 2019

matriv requested review from costin and bpintea October 9, 2019 10:02

matriv marked this pull request as ready for review October 9, 2019 10:02

Rename args, add pipe tests

b8230f3

costin approved these changes Oct 10, 2019

View reviewed changes

costin mentioned this pull request Oct 10, 2019

SQL: Refactor binary date time functions #47786

Merged

astefan approved these changes Oct 10, 2019

View reviewed changes

matriv added 3 commits October 10, 2019 13:11

Address comments

20d18ae

Added more integ tests

774b0b9

Merge remote-tracking branch 'upstream/master' into impl-add

df1862e

Fix whitespace

e9e5f57

matriv merged commit e624bc2 into elastic:master Oct 10, 2019

matriv deleted the impl-add branch October 10, 2019 13:24

jakelandis removed the v8.0.0 label Jul 26, 2021

jakelandis added the v8.0.0-alpha1 label Jul 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SQL: Implement DATEADD function #47747

SQL: Implement DATEADD function #47747

matriv commented Oct 8, 2019

elasticmachine commented Oct 8, 2019

matriv commented Oct 9, 2019

costin left a comment

costin Oct 10, 2019

costin Oct 10, 2019

costin Oct 10, 2019

costin Oct 10, 2019

costin Oct 10, 2019

matriv Oct 10, 2019

astefan left a comment

astefan Oct 9, 2019

astefan Oct 9, 2019

astefan Oct 9, 2019

astefan Oct 10, 2019

matriv Oct 10, 2019

astefan Oct 10, 2019

astefan Oct 10, 2019

astefan Oct 10, 2019

matriv Oct 10, 2019

matriv commented Oct 10, 2019


		.Description:

		Add/Subtract the given number of units to/from a date/datetime. If any of the three arguments is `null`


		Input:

		<1> string expression denoting the unit to add to the date/datetime


		public static final String NAME = "dtadd";

		public DateAddProcessor(Processor source1, Processor source2, Processor source3, ZoneId zoneId) {

SQL: Implement DATEADD function #47747

SQL: Implement DATEADD function #47747

Conversation

matriv commented Oct 8, 2019

elasticmachine commented Oct 8, 2019

matriv commented Oct 9, 2019

costin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

astefan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matriv commented Oct 10, 2019