Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hive Type cast error String to Text #64

Closed
nmaillard opened this issue Jul 24, 2013 · 1 comment
Closed

Hive Type cast error String to Text #64

nmaillard opened this issue Jul 24, 2013 · 1 comment

Comments

@nmaillard
Copy link

Hello everyone

I am using the hive to elasticsearch connector and I have come across a bug in a specific case.
Say I create a table:
CREATE EXTERNAL TABLE artists {
name STRING,
STORED BY 'org.elasticsearch.hadoop.hive.ESStorageHandler' ....

And I want to add data but my name is encoded or I need to do stuff to the name value before pushing if for example:

INSERT OVERWRITE TABLE artists select
java_method("java.net.URLDecoder", "decode", name)
from initial_table.

This use case will throw a cast exeception;
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.hadoop.io.Text
org.elasticsearch.hadoop.hive.ESSerDe.serialize(ESSerDe.java:121)

java_method returns a string but the hive artists table says a it should be a hive String aka a Hadoop text
So in HiveValueWriter it is cast to what it is expected to:
writable = ((LazyPrimitive) data).getWritableObject()

Quick fix in hive use a concat function such as:
concat("",java_method("java.net.URLDecoder", "decode", name))

But a better solution would be to check the type exposed by the hive table and the type of the data to see if something more than casting needs to be done.

I'll look into the hive code to see how they deal with the java_method when inserting into tables.

Thanks for all the good work

@costin
Copy link
Member

costin commented Jul 30, 2013

This is a duplicate of #57 (which has been fixed a couple of days ago). I've added a test which invokes a static method which works fine:
SELECT java_method(\"java.lang.System\", \"currentTimeMillis\") FROM methodInvocation LIMIT 1 returns
1375220343871

Please try the latest snapshot and report back.

@costin costin closed this as completed Jul 30, 2013
costin added a commit that referenced this issue Jul 31, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants