Mapping hive field to elastic geo_shape type

[orlevii]

Hi, I'm trying to read/write data from/to elasticsearch using Hive and I get ClassCastException when I try to read/write from/to index with geo_shape field.

I created this table in hive:

CREATE EXTERNAL TABLE features 
(
    SHAPE STRUCT<coordinates:array<array<array<double>>>,type:string>
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'hive_test/features',
              'es.index.auto.create' = 'false') ;

Elasticsearch index mapping:

PUT hive_test/features
{
    "properties":{
        "shape":{
            "type":"geo_shape"
        } 
    }
}

[costin]

The exception occurs since the mapping for geo_shape (as with all non-primitive mappings) is a string. The reason behind that is that there's no dedicated type for it in Pig and thus it's up to the user to decide how to map it.
Change your mapping in Pig from nested array to chararray and rerun your task.

[orlevii]

I'm restricted to use hive.
Is there any way to use hive or do I have to use Pig in order to map to geo_shape field?

[costin]

Neither Pig or Hive support a geo type; but that's not my point. Rather in your mapping define shape as a string and once it's loaded, then convert in Pig in whatever structure you want/need.

[orlevii]

I'm not sure I understood what you are suggesting.
But I need the mapping to be geo_shape becuase of geographic queries on the index.

[costin]

Change your mapping in Pig from nested array to chararray and rerun your task.

CREATE EXTERNAL TABLE features (SHAPE chararray)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'hive_test/features',
              'es.index.auto.create' = 'false') ;

[costin]

Closing the issue (no further follow-up).