group by error

[lu4nx]

My env:

Hive: 0.13
elasticsearch-hadoop-hive: 2.0.2:

Similar to the following HQL:

from(select ip,site from logs) output         
INSERT OVERWRITE TABLE count_logs                             
select ip,site group by ip, site;

running and Hive Thrown a error:

Caused by: java.lang.ArithmeticException: / by zero                 
        at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWrite\
r.initSingleIndex(EsOutputFormat.java:230)                          
        at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWrite\
r.init(EsOutputFormat.java:199)                                     
        at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$EsHiveR\
ecordWriter.write(EsHiveOutputFormat.java:58)                       
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processO\
p(FileSinkOperator.java:621)                                        
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator\
.java:796)                                                          
        at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(\
SelectOperator.java:87)                                             

I think EsHiveOutputFormat has a bug?

[costin]

Indeed, it looks like it. Anything special about your configuration? How are you running Hive - what distro? Is it MR or Tez by any chance?
Anything shows up in the logs with respect to Elasticsearch shards?

[lu4nx]

I using distro is CDH, and the version is 5.1.1

ElasticSearch is default settings, I just change the server's IP.

[costin]

@1u4nx Can you post your query (assuming it was created on a fresh index)? It looks like Hive is passing an invalid state to the underlying HiveOutputFormat which leads to the error you are seeing. Logs would also come in handy.

Thanks!

[costin]

Since there hasn't been any update, I'm closing the issue. Please open a new one if it appears again.