Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ORC file sampler is not able to identify the date data type #16573

Open
Anistojoseph opened this issue Jun 7, 2024 · 1 comment
Open

ORC file sampler is not able to identify the date data type #16573

Anistojoseph opened this issue Jun 7, 2024 · 1 comment
Labels

Comments

@Anistojoseph
Copy link

Please provide a detailed title (e.g. "Broker crashes when using TopN query with Bound filter" instead of just "Broker crashes").

Affected Version

We observed the issue from Druid version 26.0.0 and the same issue is there on the latest version which is 29.0.1
on Version 0.22.1 we are not seeing this issue.

Description

we encountering an issue when trying to parse the ORC file created from a Hive external table on Druid version 27.0.0. The error message "Failed to sample data: Unhandled type: COMPLEX" is displayed. However, I can load the data into the cluster using the pre-created JSON spec, and the data is being received correctly. The error only occurs during the Parse data step. Interestingly, I can parse the same file on Druid version 0.22.1.
(The JSON spec used to load the data on version 27 was created from version 0.22.1)

we have also tried the latest version (29.0.1)of Druid, but we are still encountering the same error.

in this case it is an issue with the date type. our ORC reader returns it as a Joda DateTime object. our auto type detection code is not able to figure out what to do with this. the type detection code is expecting to see more primitive types like strings, numbers, & arrays. what changed in recent druid versions is that the sampler started using auto type detection.

image
Screenshot 2024-06-07 at 4 22 04 PM

@Anistojoseph
Copy link
Author

@asdf2014 Is there any progress or update on this issue ? Which version we can expect this to be fixed ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants