You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using the hive to elasticsearch connector and I have come across a bug in a specific case.
Say I create a table:
CREATE EXTERNAL TABLE artists {
name STRING,
STORED BY 'org.elasticsearch.hadoop.hive.ESStorageHandler' ....
And I want to add data but my name is encoded or I need to do stuff to the name value before pushing if for example:
INSERT OVERWRITE TABLE artists select
java_method("java.net.URLDecoder", "decode", name)
from initial_table.
This use case will throw a cast exeception;
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.hadoop.io.Text
org.elasticsearch.hadoop.hive.ESSerDe.serialize(ESSerDe.java:121)
java_method returns a string but the hive artists table says a it should be a hive String aka a Hadoop text
So in HiveValueWriter it is cast to what it is expected to:
writable = ((LazyPrimitive) data).getWritableObject()
Quick fix in hive use a concat function such as:
concat("",java_method("java.net.URLDecoder", "decode", name))
But a better solution would be to check the type exposed by the hive table and the type of the data to see if something more than casting needs to be done.
I'll look into the hive code to see how they deal with the java_method when inserting into tables.
Thanks for all the good work
The text was updated successfully, but these errors were encountered:
This is a duplicate of #57 (which has been fixed a couple of days ago). I've added a test which invokes a static method which works fine: SELECT java_method(\"java.lang.System\", \"currentTimeMillis\") FROM methodInvocation LIMIT 1 returns
1375220343871
Hello everyone
I am using the hive to elasticsearch connector and I have come across a bug in a specific case.
Say I create a table:
CREATE EXTERNAL TABLE artists {
name STRING,
STORED BY 'org.elasticsearch.hadoop.hive.ESStorageHandler' ....
And I want to add data but my name is encoded or I need to do stuff to the name value before pushing if for example:
INSERT OVERWRITE TABLE artists select
java_method("java.net.URLDecoder", "decode", name)
from initial_table.
This use case will throw a cast exeception;
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.hadoop.io.Text
org.elasticsearch.hadoop.hive.ESSerDe.serialize(ESSerDe.java:121)
java_method returns a string but the hive artists table says a it should be a hive String aka a Hadoop text
So in HiveValueWriter it is cast to what it is expected to:
writable = ((LazyPrimitive) data).getWritableObject()
Quick fix in hive use a concat function such as:
concat("",java_method("java.net.URLDecoder", "decode", name))
But a better solution would be to check the type exposed by the hive table and the type of the data to see if something more than casting needs to be done.
I'll look into the hive code to see how they deal with the java_method when inserting into tables.
Thanks for all the good work
The text was updated successfully, but these errors were encountered: