Skip to content

Pig empty Maps #89

Closed
Closed
@nmaillard

Description

@nmaillard

Hello everyone

I have come accross this error in my pig to elsaticsearch import where having an empty map will throw a null pointer exception.

to be more precise I have a simple pigscript:

DEFINE ESStorage org.elasticsearch.hadoop.pig.ESStorage('es.host=thehost');

users = LOAD 'hbase://TABLE'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
'info:* crm:* synonymes:*', '-loadKey=true')
AS (id:chararray, info:map[], crm:map[], synonymes:map[]);

STORE usersINTO 'test/test_users' USING ESStorage();

In this situtation it may happen that some of my maps are empty because my data matrix is sparse. Say info and synonymes are ok and crm is empty some times.

Pig will throw this error:

java.lang.NullPointerException
at org.elasticsearch.hadoop.pig.PigValueWriter.write(PigValueWriter.java:103)
at org.elasticsearch.hadoop.pig.PigValueWriter.write(PigValueWriter.java:128)

looking at the line

case DataType.MAP:
ResourceSchema nestedSchema = field.getSchema();
ResourceFieldSchema[] nestedFields = nestedSchema.getFields();

I'm guessing this happens because my map is empty as as such there are no schema and fields.

regards

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions