Invalid jackson fallback on MapR distro

[costin]

Pre 1.7 jackson versions are not properly detected resulting in an invalid code path being executed. This typically occurs on MapR distros.

[beljun]

Hi. I grabbed the nightly build elasticsearch-hadoop-2.1.0.BUILD-20140617.012002-21.jar and tried to write to ES from Pig. Still getting a NoSuchMethodError when referencing the Jackson lib (MapR distro, Jackson 1.5.2). Might it be related to this same issue even after the fix?

2014-06-17 19:08:08,491 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.reader(Ljava/lang/Class;)Lorg/codehaus/jackson/map/ObjectReader;
    at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(RestClient.java:163)
    at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:156)
    at org.elasticsearch.hadoop.rest.RestRepository.sendBatch(RestRepository.java:170)
    at org.elasticsearch.hadoop.rest.RestRepository.close(RestRepository.java:189)
    at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.doClose(EsOutputFormat.java:294)
    at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.close(EsOutputFormat.java:276)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.close(PigOutputFormat.java:149)
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:561)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:670)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1126)
    at org.apache.hadoop.mapred.Child.main(Child.java:264)

[costin]

Try the 2.0.1.BUILD not the 2.1.0. Specifically https://oss.sonatype.org/content/repositories/snapshots/org/elasticsearch/elasticsearch-hadoop/2.0.1.BUILD-SNAPSHOT/

Can you also confirm the MapR version used?

Thanks,

[charlesflynn]

Hey there,

I'm also experiencing this problem. The MapR version is 2.1.3.20987.GA. I just downloaded elasticsearch-hadoop-2.0.1.BUILD-20140617.021905-22.zip and reran my Hive and Pig jobs using elasticsearch-hadoop-2.0.1.BUILD-SNAPSHOT.jar. Both returned:

2014-06-17 14:05:20,506 FATAL ExecMapper: java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.reader(Ljava/lang/Class;)Lorg/codehaus/jackson/map/ObjectReader;

[charlesflynn]

If it helps, here are the jar versions:

$HADOOP_HOME/lib:
jackson-core-asl-1.5.2.jar
jackson-mapper-asl-1.5.2.jar

$HIVE_HOME/lib:
jackson-core-asl-1.8.8.jar
jackson-jaxrs-1.8.8.jar
jackson-mapper-asl-1.8.8.jar
jackson-xc-1.8.8.jar

$PIG_HOME/lib:
jackson-core-asl-1.8.8.jar
jackson-jaxrs-1.8.8.jar
jackson-mapper-asl-1.8.8.jar
jackson-xc-1.8.8.jar

[costin]

@charlesflynn It does. Unfortunately the classpath is unreliable so on some systems the 1.8.8 version is picked up, on others 1.5.2. In your case the old version is picked up which results in the error above.
I'm working on a fix but it might take a bit longer - in the meantime you could try replacing the 1.5.2 versions with the 1.8.8 one.

[beljun]

@costin, I'm on 2.0.1.15869.GA but everything else same as what @charlesflynn pasted above. Ok, I can afford to wait some for the fix. Looking forward to it. Thanks!

[costin]

@beljun @charlesflynn guys, I've pushed a fix (and a nightly build) on the 2.0.x branch to address this issue. It works on my side and it be great if you can confirm this on your end as well.

Thanks!

[charlesflynn]

Awesome, thanks! Worked from both Hive and Pig, using elasticsearch-hadoop-2.0.1.BUILD-20140618.210714-25.jar

[beljun]

Works on my end, too. Thanks a lot, @costin !