Skip to content

Some GenericUDFs wrongly cast ObjectInspectors - #4896

Merged
ayushtkn merged 2 commits into
apache:masterfrom
okumin:HIVE-27905-split
Dec 6, 2023
Merged

ayushtkn merged 2 commits into
apache:masterfrom
okumin:HIVE-27905-split

Conversation

@okumin

@okumin okumin commented Nov 23, 2023 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

We have some UDFs that don't correctly cast ObjectInspectors.

https://issues.apache.org/jira/browse/HIVE-27905

Why are the changes needed?

Because we'd like to show better error messages. It should be helpful for users, and administrators who are monitoring errors. The original message is too primitive.

0: jdbc:hive2://hive-hiveserver2:10000/defaul> select split(array('a,b,c'), ',');
Error: Error while compiling statement: FAILED: ClassCastException org.apache.hadoop.hive.serde2.objectinspector.StandardConstantListObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector (state=42000,code=40000)

Does this PR introduce any user-facing change?

Yes. Error messages can change.

Is the change a dependency upgrade?

No.

How was this patch tested?

I added integration tests.

@okumin okumin changed the title HIVE-27905: SPLIT throws ClassCastException [WIP] HIVE-27905: SPLIT throws ClassCastException Nov 24, 2023
@okumin okumin changed the title [WIP] HIVE-27905: SPLIT throws ClassCastException HIVE-27905: SPLIT throws ClassCastException Nov 24, 2023
@okumin
okumin marked this pull request as ready for review November 24, 2023 09:00

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be a problem in other places I believe, I see similar code in GenericUDFElt/GenericUDFLocate and many other places as well,
can we fix there as well?

@okumin

okumin commented Nov 27, 2023

Copy link
Copy Markdown
Contributor Author

@ayushtkn Thanks. Can I address all similar issues in this single ticket?

@ayushtkn

Copy link
Copy Markdown
Member

Yep

@okumin
okumin marked this pull request as draft December 4, 2023 13:44
@okumin okumin changed the title HIVE-27905: SPLIT throws ClassCastException Some GenericUDFs wrongly cast ObjectInspectors Dec 4, 2023
@okumin okumin changed the title Some GenericUDFs wrongly cast ObjectInspectors [WIP] Some GenericUDFs wrongly cast ObjectInspectors Dec 4, 2023
@sonarqubecloud

sonarqubecloud Bot commented Dec 4, 2023

Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

warning The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

@okumin okumin changed the title [WIP] Some GenericUDFs wrongly cast ObjectInspectors Some GenericUDFs wrongly cast ObjectInspectors Dec 5, 2023
@okumin
okumin marked this pull request as ready for review December 5, 2023 05:03
@okumin

okumin commented Dec 5, 2023

Copy link
Copy Markdown
Contributor Author

@ayushtkn I changed the scope, checked all UDFs, and added validations to where we incorrectly cast ObjectInspectors. I checked them by my eyes and it is possible that I missed some other mistakes.
For your information, most UDFs don't raise ClassCastException. For example, GenericUDFElt has a validation before it uses ObjectInspector.

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ayushtkn
ayushtkn merged commit d6158aa into apache:master Dec 6, 2023
@okumin
okumin deleted the HIVE-27905-split branch December 6, 2023 15:07
@okumin

okumin commented Dec 6, 2023

Copy link
Copy Markdown
Contributor Author

Thank you!

tarak271 pushed a commit to tarak271/hive-1 that referenced this pull request Dec 19, 2023
…). (okumin, reviewed by Ayush Saxena, Attila Turoczy)
deniskuzZ pushed a commit to deniskuzZ/hive that referenced this pull request Jan 2, 2024
…). (okumin, reviewed by Ayush Saxena, Attila Turoczy)
dengzhhu653 pushed a commit to dengzhhu653/hive that referenced this pull request Mar 7, 2024
…). (okumin, reviewed by Ayush Saxena, Attila Turoczy)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants