Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add date to varchar coercion for hive #19500

Conversation

findinpath
Copy link
Contributor

@findinpath findinpath commented Oct 23, 2023

Description

Add date to varchar coercion for hive.

Enable the Hive users to retrieve the information from their tables after performing queries which change date columns to varchar.

Hive sample DDL query:

ALTER TABLE mytable CHANGE COLUMN mycolumn mycolumn varchar(12);

Additional context and related issues

After changing the column type, dates to far in the future are displayed as expected in Trino, but not in Hive:

0: jdbc:hive2://localhost:10000/default> select * from test2;
+-------------+
|   test2.d   |
+-------------+
| 2023-01-10  |
| 000-01-24  |
+-------------+
trino> select * from hive.default.test2;
      d       
--------------
 +10000-01-24 
 2023-01-10   
(2 rows)

In any case, dates to far in the future seem not to be queryable anyway even when working with date column type in Hive

0: jdbc:hive2://localhost:10000/default> insert into test3 values (date '10000-01-24');
0: jdbc:hive2://localhost:10000/default> select * from test3;
java.lang.IllegalArgumentException
	at java.sql.Date.valueOf(Date.java:143)
	at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:451)
	at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:425)
	at org.apache.hive.jdbc.HiveBaseResultSet.getObject(HiveBaseResultSet.java:467)
	at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:160)
	at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:57)
	at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
	at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2305)

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Hive
* Add date to varchar coercion for hive. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Oct 23, 2023
@github-actions github-actions bot added tests:hive hive Hive connector labels Oct 23, 2023
@findinpath findinpath force-pushed the findinpath/hive-date-to-varchar-coercion branch from 127a286 to 237a657 Compare October 23, 2023 20:33
@findinpath findinpath marked this pull request as ready for review October 23, 2023 20:33
@findinpath findinpath self-assigned this Oct 23, 2023
@findinpath findinpath marked this pull request as draft October 23, 2023 20:43
public void testDateToVarchar()
{
assertDateToVarcharCoercion(createUnboundedVarcharType(), LocalDate.of(2023, 1, 10), "2023-01-10");
assertDateToVarcharCoercion(createUnboundedVarcharType(), LocalDate.of(1899, 12, 31), "1899-12-31");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a thought, for opposite way conversion (VARCHAT => DATE) we only allow dates greater than 01.01.1900 is it ok to allow broader range here?

@Praveen2112 Praveen2112 force-pushed the findinpath/hive-date-to-varchar-coercion branch 2 times, most recently from 95dc3fc to 6b328f6 Compare November 6, 2023 07:35
@findinpath findinpath closed this Nov 7, 2023
@findinpath findinpath reopened this Nov 13, 2023
@findinpath findinpath force-pushed the findinpath/hive-date-to-varchar-coercion branch from 6b328f6 to e36e883 Compare November 14, 2023 15:13
@github-actions github-actions bot added the docs label Nov 14, 2023
@findinpath findinpath marked this pull request as ready for review November 14, 2023 15:13
@findinpath findinpath force-pushed the findinpath/hive-date-to-varchar-coercion branch from e36e883 to 1979724 Compare December 4, 2023 19:38
@findinpath
Copy link
Contributor Author

Unrelated CI failure #16315

@Praveen2112 Praveen2112 merged commit 33dc08f into trinodb:master Dec 12, 2023
58 of 60 checks passed
@Praveen2112
Copy link
Member

Thanks for working on this.

@github-actions github-actions bot added this to the 435 milestone Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

3 participants