Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

add date and time support #560

Merged
merged 3 commits into from Jul 13, 2020

Conversation

penghuo
Copy link
Contributor

@penghuo penghuo commented Jul 10, 2020

Description of changes

  1. Define DATE, TIME, TIMESTAMP as the core type of query engine.
  2. Add the date, time and timestamp Literal.
  3. Add the INTEGER dayofmonth(DATE) function.
  4. Add the doc. https://github.com/penghuo/sql/blob/datetype/docs/user/dql/expressions.rst

To Reviewer

  1. Missing the correct test, because the JDBC doesn't support TIMESTAMP and TIME type, and also has the wrong mapping between DATE and TIMESTMAP.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@penghuo penghuo added the SQL label Jul 10, 2020
@penghuo penghuo self-assigned this Jul 10, 2020
Copy link
Member

@chloe-zh chloe-zh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better if the value() of DATE, TIME, TIMESTAMP are made to return LocalDate, LocalTime, Instant types rather than String type? Since they should work in some arithmetic operations like calculating the date/time difference fo example DATE ‘2020-07-09’ - DATE ‘2020-07-08’, and should also work in adding and subtracting INTERVAL like https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_timestampadd

@penghuo
Copy link
Contributor Author

penghuo commented Jul 10, 2020

Would it be better if the value() of DATE, TIME, TIMESTAMP are made to return LocalDate, LocalTime, Instant types rather than String type? Since they should work in some arithmetic operations like calculating the date/time difference fo example DATE ‘2020-07-09’ - DATE ‘2020-07-08’, and should also work in adding and subtracting INTERVAL like https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_timestampadd

The value() interface has two purpose now,
(1) Used to get the original Java Data and then perform the calculation.
(2) Used by Protocol to get the value for result.
For primitive data type like integer, string, it works fine. But for other types, like date, the format should be controlled by the value itself.
Currently, for ExprDateValue, the value() interface is used by the Protocol to get the serialized result, and I add getDate() method in ExprDateValue performing calculation.
I am also thinking deprecated the value interface and add integer(), long(), and date(), time() interface instead.

Copy link
Member

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes!

@penghuo penghuo merged commit 4b33a2f into opendistro-for-elasticsearch:master Jul 13, 2020
penghuo added a commit that referenced this pull request Jul 13, 2020
penghuo added a commit that referenced this pull request Jul 13, 2020
chloe-zh added a commit that referenced this pull request Aug 20, 2020
* Bug fix, support long type for aggregation (#522)

* Bug fix, support long type for aggregation

* change to datetime to JDBC format

* Opendistro Release 1.9.0 (#532)

* prepare odfe 1.9

* Fix all ES 7.8 compile and build errors

* Revert changes as Lombok is working now

* Update CustomExternalTestCluster.java

* Fix license headers check

* Use splitFieldsByMetadata to separate fields when calling SearchHit constructor

* More fixes for ODFE 1.9

* Remove todo statement

* Add ODFE 1.9.0 release notes

* Rename release notes to use 4 digit versions (#547)

* Revert changes ahead of develop branch in master (#551)

* Revert "Rename release notes to use 4 digit versions (#547)"

This reverts commit 33c6d3e.

* Revert "Opendistro Release 1.9.0 (#532)"

This reverts commit 254f2e0.

* Revert "Bug fix, support long type for aggregation (#522)"

This reverts commit fb2ed91.

* Merge all SQL repos and adjust workflows (#549) (#554)

* merge all sql repos

* fix test and build workflows

* fix workbench and odbc path

* fix workbench and odbc path

* restructure workbench dir and fix workflows

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* revert workbench directory structure

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* update workbench workflow for release

* Delete .github/ in sql-workbench directory

* Add cypress to sql-workbench

* Sync latest ODBC commits

* Sync latest workbench commits (will add cypress in separate PR)

* Add ignored ODBC libs

* add date and time support (#560)

* add date and time support

* update doc

* update doc

* Revert "add date and time support (#560)" (#567)

This reverts commit 4b33a2f.

* add error details for all server communication errors (#645)

- add null check to avoid crashing if details not initialized

* Revert "add error details for all server communication errors (#645)" (#653)

This reverts commit c11125d.

* upgrade to es7.9.0 and kibana7.9.0

* update lockfile

Co-authored-by: Peng Huo <penghuo@gmail.com>
Co-authored-by: Joshua <joshuali925@gmail.com>
Co-authored-by: Joshua Li <lijshu@amazon.com>
Co-authored-by: Jordan Wilson <37088125+jordanw-bq@users.noreply.github.com>
chloe-zh added a commit that referenced this pull request Sep 24, 2020
…rim, trim, upper, lower, concat, concat_ws, length, strcmp (#750)

* Bug fix, support long type for aggregation (#522)

* Bug fix, support long type for aggregation

* change to datetime to JDBC format

* Opendistro Release 1.9.0 (#532)

* prepare odfe 1.9

* Fix all ES 7.8 compile and build errors

* Revert changes as Lombok is working now

* Update CustomExternalTestCluster.java

* Fix license headers check

* Use splitFieldsByMetadata to separate fields when calling SearchHit constructor

* More fixes for ODFE 1.9

* Remove todo statement

* Add ODFE 1.9.0 release notes

* Rename release notes to use 4 digit versions (#547)

* Revert changes ahead of develop branch in master (#551)

* Revert "Rename release notes to use 4 digit versions (#547)"

This reverts commit 33c6d3e.

* Revert "Opendistro Release 1.9.0 (#532)"

This reverts commit 254f2e0.

* Revert "Bug fix, support long type for aggregation (#522)"

This reverts commit fb2ed91.

* Merge all SQL repos and adjust workflows (#549) (#554)

* merge all sql repos

* fix test and build workflows

* fix workbench and odbc path

* fix workbench and odbc path

* restructure workbench dir and fix workflows

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* revert workbench directory structure

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* update workbench workflow for release

* Delete .github/ in sql-workbench directory

* Add cypress to sql-workbench

* Sync latest ODBC commits

* Sync latest workbench commits (will add cypress in separate PR)

* Add ignored ODBC libs

* add date and time support (#560)

* add date and time support

* update doc

* update doc

* Revert "add date and time support (#560)" (#567)

This reverts commit 4b33a2f.

* add error details for all server communication errors (#645)

- add null check to avoid crashing if details not initialized

* Revert "add error details for all server communication errors (#645)" (#653)

This reverts commit c11125d.

* Fix download link in package description (#729)

* [1] Initial commit, checking if server build passes

* [1] Commiting expression documentation with REGEXP

* [1] Failure with REGEXP doc

* [1] Moved testing for regexp to binary predicates.

* [1] Updating parser for REGEX

* [1] Parser update

* [1] Making REGEXP like LIKE

* [1] Reverting change to legacy

* [1] Checking if same without NOT

* [1] testing adding to Ast Expr

* [1] Switching REGEX over to Integer.

* [1] Reversion test

* [1] Add back test

* [1] Fixing spacing

* [1] Regexp builder test.

* -2

* -1

* [1] trying with semicolon

* [1] Found the missing link >_<

* [1] Functions documentation

* [1] Fixing documentation mistake.

* [1] Retesting

* [1] Trying to debug python

* [1] more python debug info

* [1] Trying again.

* [1] MOre py inof

* [1] Fixed except

* [1] Simplified concat and concat_ws

* [1] Added missing stuff to paraser

* [1] Fixed some functions and removed some unused imports

* [1] Fixed STRCMP

* [1] Trying to fix aliasing issue with substring

* [1] Fixed stringcompare and substring

* [1] REmoving unused imorts

* [1] Fixed documentation

* [1] REGEXP not supported by sqllite so removing these for now.

* [1] Removed auto IT and added manual.

* [1] Fixed spacing

* [1] Fixed type definitions

* [1] Fixed integer values and ltrim

* [1] COrrecting ltrim again

* [1] Changed patterns

* [1] Fixed some minor issues

* [1] reverting change i didnt make

* [1] Condensed logic
[2] Added EMPTY_STRING
[3] Removed unused function resolver

* [1] Removed SUBSTRING FunctionName.

* [1] Reverted failure issues

* [1] Combined substring and substr test.

* [1] Added ppl test and edited caps in textfunctiontest

* [1] Testing without source

* [1] Correcting format of string

* [1] Testing new queries

* [1] Adding resource and fixed tests

* [1] Added maapping and adjusted tests

* [1] minor corrections

* [1] Additional debug info

* [1] Removing unsuspported ppl functions.

* [1] Added back unsupported

* [1] Checking regex

* [1] Removing printout

* [1] Trying to fix commit

* Revert "[1] Trying to fix commit"

This reverts commit 3fa954c.

* [1] Adding rest of commit

* [1] Adding workflow files

* [1] fixed docs

* [1] Updated based on PR comments.

* [1] Fixed code so tests pass

Co-authored-by: Peng Huo <penghuo@gmail.com>
Co-authored-by: Joshua <joshuali925@gmail.com>
Co-authored-by: Joshua Li <lijshu@amazon.com>
Co-authored-by: Jordan Wilson <37088125+jordanw-bq@users.noreply.github.com>
Co-authored-by: Chloe <chloezh1102@gmail.com>
Co-authored-by: chloe-zh <fizhang@amazon.com>
Co-authored-by: Sayali Gaikawad <61760125+gaiksaya@users.noreply.github.com>
penghuo added a commit that referenced this pull request Oct 2, 2020
* Bug fix, support long type for aggregation (#522)

* Bug fix, support long type for aggregation

* change to datetime to JDBC format

* Opendistro Release 1.9.0 (#532)

* prepare odfe 1.9

* Fix all ES 7.8 compile and build errors

* Revert changes as Lombok is working now

* Update CustomExternalTestCluster.java

* Fix license headers check

* Use splitFieldsByMetadata to separate fields when calling SearchHit constructor

* More fixes for ODFE 1.9

* Remove todo statement

* Add ODFE 1.9.0 release notes

* Rename release notes to use 4 digit versions (#547)

* Revert changes ahead of develop branch in master (#551)

* Revert "Rename release notes to use 4 digit versions (#547)"

This reverts commit 33c6d3e.

* Revert "Opendistro Release 1.9.0 (#532)"

This reverts commit 254f2e0.

* Revert "Bug fix, support long type for aggregation (#522)"

This reverts commit fb2ed91.

* Merge all SQL repos and adjust workflows (#549) (#554)

* merge all sql repos

* fix test and build workflows

* fix workbench and odbc path

* fix workbench and odbc path

* restructure workbench dir and fix workflows

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* revert workbench directory structure

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* fix workbench workflow

* update workbench workflow for release

* Delete .github/ in sql-workbench directory

* Add cypress to sql-workbench

* Sync latest ODBC commits

* Sync latest workbench commits (will add cypress in separate PR)

* Add ignored ODBC libs

* add date and time support (#560)

* add date and time support

* update doc

* update doc

* Revert "add date and time support (#560)" (#567)

This reverts commit 4b33a2f.

* add error details for all server communication errors (#645)

- add null check to avoid crashing if details not initialized

* Revert "add error details for all server communication errors (#645)" (#653)

This reverts commit c11125d.

* Fix download link in package description (#729)

* add functions day, month, quarter, year

* fix build error

* fix doctest error

* fix doctest build error

* fix doctest

* add dayofmonth()

* add  dayofyear()

* add dayofweek()

* fix dayofweek logic & add unit test

* fix doctest for dayofweek()

* add dayname

* add monthname

* fix checkstyle build error

* fix build error

* fix doctest for monthname

* add hour()

* add minute()

* add second

* add microsecond

* fix datetime & timestamp issue for microsecond

* add time_to_sec

* add subdate & date_sub

* fix doctest error

* fix build error

* add KeywordsCanBeId for dayofweek

* add to_days

* add from_days()

* arrange by alphabetical order

* add manual IT

* add string input for date functions

* fix microsecond

* update doc

* add date_add

* add week

* address PR comments

* update tests & doc

* fix doc format

* update tests for adddate

* move string conversion to ExprStringValue

* add string type in doc

* edge case

* fix case 5 & 7

* add IT

* update doc

* fix table

* rename

* add string type

* nit: add newline

* fix type in comment

* nit

* add test cases for datetime function in ExpeStringValue

* removing implicit def for keyword in parser

* add dayofweek

* [1] Merged rupals week branch in

* add unit tests for null, missing values

* nit

* [1] Added integration tests.

* [1] Working on ppl integration tests.

* [1] Updated for integration tests

* [1] Fixed schema verification

* [1] Adding documentation.

* [1] Fixing documentation

* [1] Simplified a bunch of logic

* [1] Reducing changes that are from spacing

* [1] Removed extra merge line

* address PR comment

* [1] Updates

* [1] Removed some unwanted changes.

* [1] Minor whitespace adjustements

* [1] Updating based on code review

Co-authored-by: Peng Huo <penghuo@gmail.com>
Co-authored-by: Joshua <joshuali925@gmail.com>
Co-authored-by: Joshua Li <lijshu@amazon.com>
Co-authored-by: Jordan Wilson <37088125+jordanw-bq@users.noreply.github.com>
Co-authored-by: Chloe <chloezh1102@gmail.com>
Co-authored-by: chloe-zh <fizhang@amazon.com>
Co-authored-by: Sayali Gaikawad <61760125+gaiksaya@users.noreply.github.com>
Co-authored-by: Rupal Mahajan <>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants