Repository navigation
HIVE-28075: Vectorized DayOFWeek returns inconsistent results for non-UTC timezones - #5837
Conversation
1a111fe to
ba01e6e
Compare
asf-ci-hive
added
tests failed
tests pending
tests passed
and removed
tests pending
tests failed
tests passed
labels
…ith non-vectorized version
| date = format.parse(decoded); | ||
| Date date = DateParser.parseDate(decoded.toString()); | ||
| if (date == null) { | ||
| throw new ParseException("Unable to parse date string: " + decoded, 0); |
There was a problem hiding this comment.
I honestly don't know the exact security policy of Hive, and I may be overthinking it. When I examined the relevant codes, I found that no piece had exposed the argument or row. Is it possible that we should not expose decoded? Say a malicious user submits SELECT DAY_OF_WEEK(credit_card_number) FROM pii_table. A card number can be stored in HS2's log files or a client system, such as Apache Airflow. I'm considering what semantics are desirable. Related:
https://issues.apache.org/jira/browse/HIVE-20644
There was a problem hiding this comment.
Yes, this could have led to a potential CVE. i'll remove the decoded string being exposed in the exception.
Thanks for catching this @okumin !
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
okumin
left a comment
okumin
left a comment
There was a problem hiding this comment.
I added a minor comment about serialVersionUID. Other than that, looks good!
| */ | ||
| public final class VectorUDFDayOfWeekString extends VectorUDFTimestampFieldString { | ||
|
|
||
| private static final long serialVersionUID = 1L; |
There was a problem hiding this comment.
Sorry, please restore this variable because this class is serializable.
There was a problem hiding this comment.
Thanks. I will merge this after the 24-hour rule is satisfied.
|
okumin
changed the title
|
Merged. @tanishq-chugh Thanks for addressing the data integrity problem 👍 |
3 participants
…-UTC timezones
What changes were proposed in this pull request?
Fix the issue of UDF dayofweek returning inconsistent results for non-UTC time zones.
Why are the changes needed?
Currently, the Calender object being used in VectorUDFDayOfWeekString.java is initialised to use UTC timezone by default. So, the date string passed to the same is represented in UTC timezone irrespective of the timezone set in hive/JVM.
Does this PR introduce any user-facing change?
No, it is fixing a bug.
How was this patch tested?
Manual. Added a qtest as well.