Pig-ES Integration for Nested objects

[ayanguha]

Hi

I am facing an issue, probably missing very basic stuff.

DEFINE EsStorage org.elasticsearch.hadoop.pig.EsStorage('es.nodes=10.99.12.162',
                                'es.port=9200',
                                'es.mapping.date.rich=false',
                                'es.index.auto.create = false',
                                                        'es.mapping.id = connote');


a = foreach b generate TOMAP('addrLine1',Shpadd, 'addrLine2',Shpadd, 'state',Shp_st, 'suburb',Shpcity, 'country',Shpzip, 'postCode', Shpzip) as senderLocation;

store a into resource using ESStorage()

My target mapping is

"senderLocation" : {
            "properties" : {
              "addrLine1" : {
                "type" : "string"
              },
              "addrLine2" : {
                "type" : "string"
              },
              "country" : {
                "type" : "string"
              },
              "fullAddress" : {
                "type" : "string"
              },
              "mapLink" : {
                "type" : "string"
              },
              "poBox" : {
                "type" : "string"
              },
              "postCode" : {
                "type" : "string"
              },
              "state" : {
                "type" : "string"
              },
              "suburb" : {
                "type" : "string"
              },
              "unitNo" : {
                "type" : "string"
              }
            }
          },

However, when I am running the program, it does not load.

senderLocation:{}

Any idea will help!!!

[costin]

Take a look at the documentation in particular the code examples, like this one.

You don't need to create a Map - in fact, Pig and Hive tend to not have great support for these.
Simply remove it, double check your variable/tuple A has the fields you need and store that into the Storage.

Cheers,

P.S. For questions, the forum is recommended instead of the issue tracker.

[ayanguha]

Thanks Costin. I will use forum going forward.

I have resolved the problem by using tuples and
es.mapping.pig.tuple.use.field.names

On Fri, Aug 14, 2015 at 5:44 AM, Costin Leau notifications@github.com
wrote:

Take a look at the documentation in particular the code examples
https://www.elastic.co/guide/en/elasticsearch/hadoop/current/pig.html,
like this one
https://www.elastic.co/guide/en/elasticsearch/hadoop/current/pig.html#_writing_data_to_elasticsearch_3
.

You don't need to create a Map - in fact, Pig and Hive tend to not have
great support for these.
Simply remove it, double check your variable/tuple A has the fields you
need and store that into the Storage.

Cheers,

P.S. For questions, the forum http://discuss.elastic.co is recommended
instead of the issue tracker.

—
Reply to this email directly or view it on GitHub
#534 (comment)
.

Best Regards,
Ayan Guha

[costin]

Glad to hear it. Closing the issue.