Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapper: Failure to handle null valued objects ("obj1" : null) #341

Closed
an2deg opened this issue Aug 30, 2010 · 6 comments

Comments

@an2deg
Copy link

commented Aug 30, 2010

When index some of my documents I got java.lang.NullPointerException (some of documents has been indexed successfully ). There is because in some documents ['serialized_object']['avatars'] is null and in some this field is empty object (and few have not empty objects)

For index "contacts" I have install the following mapping:

es_index_mapping = {
"contact": {
"properties" : {
"id": {
"type": "string",
"store": "yes"
},
"owner_id": {
"type": "string",
"index": "not_analyzed",
"store": "yes"
},
"company_id": {
"type": "string",
"index": "not_analyzed",
"store": "yes"
},
"serialized_object": {
"type": "object",
"index": "not_analyzed",
"store": "yes",
# Do not parse fields['dates'] as date b/c its often contains a noise
# and ES prevent to index entire objects
"properties" : {
"fields": {
"type": "object",
"properties" : {
"dates": {
"type": "object",
"properties" : {
"value": {
"type": "string"
}
}
}
}
}
}
},
"fields": {
"type": "object",
"store": "yes",
# Do not parse fields.dates as date b/c its often contains a noise
# and ES prevent to index entire objects
"properties" : {
"dates": {
"type": "string",
}
}
},
"is_account": {
"type": "boolean",
"store": "yes"
},
"updated" : {
"type" : "date",
"store": "yes",
"index" : "not_analyzed",
"format" : "yyyy-MM-dd'T'HH:mm:ssZ"
},
}
}
}

Here a traceback:
[1a5e7e97-67e5-4344-bc55-828c704c5419], [P], s[STARTED]: Failed to execute [index {[contacts][contact][4bf571e8304e747bb9000081], source[{"serialized_object": {"updated": "2010-08-30T15:43:06+0000 , "account_id": null, "tags": [], "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": [{"note": "", "modificator": "", "value": "sergei", "sequenc ": 0}], "phone": [], "company_name": [], "address": [], "email": [{"note": "", "modificator": "other", "value": "sergei@kovbasyuk.com.ua", "sequence": 0}, {"note": "", "modificator": "home", "valu ": "webmans2@gmail.com", "sequence": 1}]}, "object_type": "contact", "avatars": null, "is_account": false, "company_id": "4bec433c304e745ad3000009", "id": "4bf571e8304e747bb9000081", "owner_id": " bec433c304e745ad3000007"}, "updated": "2010-08-30T15:43:06+0000", "account_id": null, "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": ["sergei , "1ZjHE5vngEUsRvj0J"], "phone": [], "company_name": [], "address": [], "email": ["sergei@kovbasyuk.com.ua", "webmans2@gmail.com", "1ZjHE5vngEUsRvj0J"], "name": ["sergei", "1ZjHE5vngEUsRvj0J"]}, " ompany_id": "4bec433c304e745ad3000009", "is_account": false, "id": "4bf571e8304e747bb9000081", "owner_id": "4bec433c304e745ad3000007"}]}]
java.lang.NullPointerException
at org.elasticsearch.index.mapper.FieldMapper$Names.(FieldMapper.java:50)
at org.elasticsearch.index.mapper.xcontent.XContentFieldMapper$Builder.buildNames(XContentFieldMapper.java:176)
at org.elasticsearch.index.mapper.xcontent.XContentStringFieldMapper$Builder.build(XContentStringFieldMapper.java:69)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.serializeValue(XContentObjectMapper.java:440)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.parse(XContentObjectMapper.java:336)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.serializeValue(XContentObjectMapper.java:405)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.serializeArray(XContentObjectMapper.java:397)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.parse(XContentObjectMapper.java:330)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.serializeObject(XContentObjectMapper.java:357)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.parse(XContentObjectMapper.java:328)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.serializeObject(XContentObjectMapper.java:357)
at org.elasticsearch.index.mapper.xcontent.XContentObjectMapper.parse(XContentObjectMapper.java:328)
at org.elasticsearch.index.mapper.xcontent.XContentDocumentMapper.parse(XContentDocumentMapper.java:362)
at org.elasticsearch.index.mapper.xcontent.XContentDocumentMapper.parse(XContentDocumentMapper.java:312)
at org.elasticsearch.index.shard.service.InternalIndexShard.innerIndex(InternalIndexShard.java:227)
at org.elasticsearch.index.shard.service.InternalIndexShard.index(InternalIndexShard.java:219)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:133)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:60)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:381)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.access$400(TransportShardReplicationOperationAction.java:208)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:278)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

@kimchy

This comment has been minimized.

Copy link
Member

commented Aug 30, 2010

Is there a chance for a "curl" recreation of this bug? Easier for me to write a test that recreates it then, and make sure its fixed.

@an2deg

This comment has been minimized.

Copy link
Author

commented Aug 31, 2010

Can't reproduce it with curl, but here it:

curl -XPUT 'http://localhost:9200/contacts3/'
curl -XPUT 'http://localhost:9200/contacts3/contact3/_mapping' -d '
{"contact3": {"properties": {"serialized_object": {"index": "not_analyzed", "type": "object", "properties": {"fields": {"type": "object", "properties": {"dates": {"type": "object", "properties": {"value": {"type": "string"}}}}}}, "store": "yes"}, "fields": {"type": "object", "properties": {"dates": {"type": "string"}}, "store": "yes"}, "company_id": {"index": "not_analyzed", "type": "string", "store": "yes"}, "is_account": {"type": "boolean", "store": "yes"}, "id": {"type": "string", "store": "yes"}, "owner_id": {"index": "not_analyzed", "type": "string", "store": "yes"}}}}'

curl -XPUT 'http://localhost:9200/contacts3/contact3/4bf571e8304e747bb9000081' -d '
{"serialized_object": { "account_id": null, "tags": [], "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": [{"note": "", "modificator": "", "value": "sergei", "sequenc ": 0}], "phone": [], "company_name": [], "address": [], "email": [{"note": "", "modificator": "other", "value": "sergei@kovbasyuk.com.ua", "sequence": 0}, {"note": "", "modificator": "home", "value": "webmans2@gmail.com", "sequence": 1}]}, "updated": "2010-08-30 15:43:06+0000", "object_type": "contact", "avatars": null, "is_account": false, "company_id": "4bec433c304e745ad3000009", "id": "4bf571e8304e747bb9000081", "owner_id": " bec433c304e745ad3000007"}, "account_id": null, "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": ["sergei", "1ZjHE5vngEUsRvj0J"], "phone": [], "company_name": [], "address": [], "email": ["sergei@kovbasyuk.com.ua", "webmans2@gmail.com", "1ZjHE5vngEUsRvj0J"], "name": ["sergei", "1ZjHE5vngEUsRvj0J"], "avatars": null}, "company_id": "4bec433c304e745ad3000009", "is_account": false, "id": "4bf571e8304e747bb9000081", "owner_id": "4bec433c304e745ad3000007", "updated": "2010-08-30 15:43:06+0000"}
'

@kimchy

This comment has been minimized.

Copy link
Member

commented Aug 31, 2010

can you recreate this in a different way so I can easily run it?

@an2deg

This comment has been minimized.

Copy link
Author

commented Aug 31, 2010

Ok, here is updated test case:

  1. Index first document with "avatars": {}
  2. Then index document with "avatars": null and get java.lang.NullPointerException

curl -XDELETE 'http://localhost:9200/contacts3/'

curl -XPUT 'http://localhost:9200/contacts3/'

curl -XPUT 'http://localhost:9200/contacts3/contact3/_mapping' -d '
{"contact3": {"properties": {"serialized_object": {"index": "not_analyzed", "type": "object", "properties": {"fields": {"type": "object", "properties": {"dates": {"type": "object", "properties": {"value": {"type": "string"}}}}}}, "store": "yes"}, "fields": {"type": "object", "properties": {"dates": {"type": "string"}}, "store": "yes"}, "company_id": {"index": "not_analyzed", "type": "string", "store": "yes"}, "is_account": {"type": "boolean", "store": "yes"}, "id": {"type": "string", "store": "yes"}, "owner_id": {"index": "not_analyzed", "type": "string", "store": "yes"}}}}'

curl -XPUT 'http://localhost:9200/contacts3/contact3/4bf571e8304e747bb9000081' -d '
{"serialized_object": { "account_id": null, "tags": [], "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": [{"note": "", "modificator": "", "value": "sergei", "sequenc ": 0}], "phone": [], "company_name": [], "address": [], "email": [{"note": "", "modificator": "other", "value": "sergei@kovbasyuk.com.ua", "sequence": 0}, {"note": "", "modificator": "home", "value": "webmans2@gmail.com", "sequence": 1}]}, "updated": "2010-08-30 15:43:06+0000", "object_type": "contact", "avatars": {}, "is_account": false, "company_id": "4bec433c304e745ad3000009", "id": "4bf571e8304e747bb9000081", "owner_id": " bec433c304e745ad3000007"}, "account_id": null, "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": ["sergei", "1ZjHE5vngEUsRvj0J"], "phone": [], "company_name": [], "address": [], "email": ["sergei@kovbasyuk.com.ua", "webmans2@gmail.com", "1ZjHE5vngEUsRvj0J"], "name": ["sergei", "1ZjHE5vngEUsRvj0J"], "avatars": {}}, "company_id": "4bec433c304e745ad3000009", "is_account": false, "id": "4bf571e8304e747bb9000081", "owner_id": "4bec433c304e745ad3000007", "updated": "2010-08-30 15:43:06+0000"} '

curl -XPUT 'http://localhost:9200/contacts3/contact3/4bf571e8304e747bb9000082' -d '
{"serialized_object": { "account_id": null, "tags": [], "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": [{"note": "", "modificator": "", "value": "sergei", "sequenc ": 0}], "phone": [], "company_name": [], "address": [], "email": [{"note": "", "modificator": "other", "value": "sergei@kovbasyuk.com.ua", "sequence": 0}, {"note": "", "modificator": "home", "value": "webmans2@gmail.com", "sequence": 1}]}, "updated": "2010-08-30 15:43:06+0000", "object_type": "contact", "avatars": null, "is_account": false, "company_id": "4bec433c304e745ad3000009", "id": "4bf571e8304e747bb9000082", "owner_id": " bec433c304e745ad3000007"}, "account_id": null, "fields": {"website": [], "dates": [], "last_name": [], "description": [], "title": [], "first_name": ["sergei", "1ZjHE5vngEUsRvj0J"], "phone": [], "company_name": [], "address": [], "email": ["sergei@kovbasyuk.com.ua", "webmans2@gmail.com", "1ZjHE5vngEUsRvj0J"], "name": ["sergei", "1ZjHE5vngEUsRvj0J"], "avatars": null}, "company_id": "4bec433c304e745ad3000009", "is_account": false, "id": "4bf571e8304e747bb9000082", "owner_id": "4bec433c304e745ad3000007", "updated": "2010-08-30 15:43:06+0000"} '

On last command I get {"error":"ReplicationShardOperationFailedException[[contacts3][1] ]; nested: "} in console and traceback in logs

@kimchy

This comment has been minimized.

Copy link
Member

commented Aug 31, 2010

found the problem, writing a test for it and will push a fix shortly

@kimchy

This comment has been minimized.

Copy link
Member

commented Aug 31, 2010

Mapper: Failure to handle null valued objects ("obj1" : null), closed by a3efa21.

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.