Skip to content

HIVE-27741 handle Invalid timezone value in to_utc_timestamp() UDF - #5014

Merged
deniskuzZ merged 1 commit into
apache:masterfrom
zratkai:HIVE-27741
Apr 9, 2024
Merged

deniskuzZ merged 1 commit into
apache:masterfrom
zratkai:HIVE-27741

Conversation

@zratkai

@zratkai zratkai commented Jan 18, 2024

Copy link
Copy Markdown
Contributor

Change-Id: I68ee1783614b4b74093df0dddb613fa860ed202a

What changes were proposed in this pull request?

The to_utc_timestamp() UDF used UTC when the given timezone was invalid. It throws exception in this case to avoid data inconsistency.

Why are the changes needed?

To avoid data inconsistency in case of invalid timezone.

Does this PR introduce any user-facing change?

Yes. The to_utc_timestamp() UDF will throw exception in case of invalid timezone.

Is the change a dependency upgrade?

No.

How was this patch tested?

Unit test.

@aturoczy aturoczy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is fine. Need a test re-run and green output.

Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated

@zratkai zratkai left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the comments and fixes.

Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated

@zratkai zratkai left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check my comments, changes.

Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
@zratkai

zratkai commented Feb 5, 2024

Copy link
Copy Markdown
Contributor Author

@okumin @simhadri-g could you please review?

@zratkai

zratkai commented Feb 7, 2024

Copy link
Copy Markdown
Contributor Author

@okumin @simhadri-g could you please review? Refactored to use ZoneId instead of TimeZone.

@simhadri-g simhadri-g left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@okumin okumin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 with some minor cosmetic comments.
This could be an incompatible change. But I understand the current behavior is surprising and it sounds better to throw an error(or put NULL at least). The final judgment will be done by a committer.

Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated
@sonarqubecloud

sonarqubecloud Bot commented Mar 7, 2024

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@zratkai

zratkai commented Mar 12, 2024

Copy link
Copy Markdown
Contributor Author

@okumin @simhadri-g could you please review this?

@okumin

okumin commented Mar 13, 2024

Copy link
Copy Markdown
Contributor

Looks good to me. Note that you finally need approval from a committer

Comment thread ql/src/test/queries/clientpositive/udf_from_utc_timestamp.q
Comment thread ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUtcTimestamp.java Outdated

@deniskuzZ deniskuzZ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, minor code-style comments

…row exception instead a default value

Change-Id: I68ee1783614b4b74093df0dddb613fa860ed202a

@deniskuzZ deniskuzZ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, pending tests

@sonarqubecloud

sonarqubecloud Bot commented Apr 9, 2024

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@deniskuzZ
deniskuzZ merged commit 28ba6fb into apache:master Apr 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants