-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issues with Julian/Gregorian transition #831
Comments
What is the use case for this? Who is actually using such old dates? |
They used for storing dates of documents from archives, for example. Also Julian calendar is still used in some religious organizations. Code for proleptic Gregorian calendar also needed for ODBC server, because Postgre uses proleptic Gregorian calendar in network level too. I wrote a class that can handle these calendar systems and I fixed old dates before transition for ODBC driver with it in my local copy. Now I'm trying to find a more complete solution. There was issue #644 about year |
I'm ok with this, as long as it is explicitly configured i.e. should default to off |
BTW, SQL Standard also has only the Gregorian calendar in its specifications. So H2's behavior seems to be incompatible with the Standard, with PostgreSQL, MySQL, SQL Server, and others, and only partially compatible with Oracle (Oracle accepts everything from Julian and Gregorian, but uses traditional mixed calendar in date-time arithmetic). |
…lex types ### What changes were proposed in this pull request? When date and timestamp values are fields of arrays, maps, etc, we convert them to hive string using `toString`. This makes the result wrong before the default transition ’1582-10-15‘. https://bugs.openjdk.java.net/browse/JDK-8061577?focusedCommentId=13566712&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13566712 cases to reproduce: ```sql +-- !query 47 +select array(cast('1582-10-13' as date), date '1582-10-14', date '1582-10-15', null) +-- !query 47 schema +struct<array(CAST(1582-10-13 AS DATE), DATE '1582-10-14', DATE '1582-10-15', CAST(NULL AS DATE)):array<date>> +-- !query 47 output +[1582-10-03,1582-10-04,1582-10-15,null] + + +-- !query 48 +select cast('1582-10-13' as date), date '1582-10-14', date '1582-10-15' +-- !query 48 schema +struct<CAST(1582-10-13 AS DATE):date,DATE '1582-10-14':date,DATE '1582-10-15':date> +-- !query 48 output +1582-10-13 1582-10-14 1582-10-15 ``` other refencences h2database/h2database#831 ### Why are the changes needed? bug fix ### Does this PR introduce any user-facing change? yes, complex types containing datetimes in `spark-sql `script and thrift server can result same as self-contained spark app or `spark-shell` script ### How was this patch tested? add uts Closes #26942 from yaooqinn/SPARK-30301. Authored-by: Kent Yao <yaooqinn@hotmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
…lex types When date and timestamp values are fields of arrays, maps, etc, we convert them to hive string using `toString`. This makes the result wrong before the default transition ’1582-10-15‘. https://bugs.openjdk.java.net/browse/JDK-8061577?focusedCommentId=13566712&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13566712 cases to reproduce: ```sql +-- !query 47 +select array(cast('1582-10-13' as date), date '1582-10-14', date '1582-10-15', null) +-- !query 47 schema +struct<array(CAST(1582-10-13 AS DATE), DATE '1582-10-14', DATE '1582-10-15', CAST(NULL AS DATE)):array<date>> +-- !query 47 output +[1582-10-03,1582-10-04,1582-10-15,null] + + +-- !query 48 +select cast('1582-10-13' as date), date '1582-10-14', date '1582-10-15' +-- !query 48 schema +struct<CAST(1582-10-13 AS DATE):date,DATE '1582-10-14':date,DATE '1582-10-15':date> +-- !query 48 output +1582-10-13 1582-10-14 1582-10-15 ``` other refencences h2database/h2database#831 bug fix yes, complex types containing datetimes in `spark-sql `script and thrift server can result same as self-contained spark app or `spark-shell` script add uts Closes apache#26942 from yaooqinn/SPARK-30301. Authored-by: Kent Yao <yaooqinn@hotmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
This issue related only for old dates, so for most users it does not relevant at all.
There is some common agreement that transition from Julian to Gregorian calendar system was after the last Juilan day
1582-10-04
followed by Gregorian1582-10-15
. However, this is not true for some countries (and not only a countries, there are also some local decisions about it). Oldjava.util.GregorianCalendar
in Java supports different transitions viasetGregorianChange()
. JDBC have a methods to deal with different calendar systems and H2 supports them inPreparedStatement.setDate(..., Calendar)
and other methods. But not in string literals. H2 accepts, for example,1300-02-29
that valid with such agreement and not valid in proleptic Gregorian calendar, but rejects dates from1582-10-05
to1582-10-14
that is not valid with such agreement, but valid in some countries and in proleptic Gregorian. So it's not possible to set them in normal way in DB console.Modern Java time API from JSR-310 supports only proleptic Gregorian calendar for some reasons.
Also H2 database always use default transition rules in assorted date-time calculations.
I checked behaviour of some other databases.
Oracle silently accepts dates from Julian and Gregorian calendars. But it thinks that difference between
1300-02-28
and1300-03-01
is 2 days and difference between1582-10-04
and1582-10-15
is 1 day, so Oracle uses the same common transition rules in such comptations. But at least it allows to set dates like1582-10-05
in literals. However, in its JDBC driver conversions of dates before1582-10-15
tojava.sql
types is broken.SQLite
accepts dates from both calendars, but this database does not care about anything.MySQL, MariaDB (fork of MySQL), PostgreSQL, and MS SQL Server reject date
1300-02-29
(I check only literals), but accept dates from1582-10-05
to1582-10-14
. Thay also return difference between1300-02-28
and1300-03-01
as 1 day and difference between1582-10-04
and1582-10-15
as 11 days. This behaviour is valid for proleptic Gregorian calendar, not for traditional one.I think that we need a some compatibility option to use other dates Juilan/Gregorian transitions including proleptic Julian and proleptic Gregorian. It is supported by
GregorianCalendar
anyway. I can implement such option if there are no complaints with it.I also think that H2 should accept dates from both Julian and Gregorian calendar systems in string literals. But I don't know if H2 should just accept them without transition check (something like Oracle), or check configured transition date for safety.
The text was updated successfully, but these errors were encountered: