NPE when the JSON to index doesn't match the mapping's expectations

[tsuna]

Here's a simple repro (works both on 0.15.2 and on HEAD as of 70fc8d9):

$ curl -XPOST 'localhost:9200/test/foo' -d '{"field": {"nested": 42}}'
{"ok":true,"_index":"test","_type":"foo","_id":"hHtWqld0RTeRMpQxNeX7bA","_version":1}
$ curl -XPOST 'localhost:9200/test/foo' -d '{"field": "omg"}'         
{"error":"NullPointerException[null]","status":500}

This is on a fresh install of 0.15.2 with no configuration whatsoever. For reference, the dynamic mapping created by the first POST is:

$ curl 'localhost:9200/test/foo/_mapping?pretty=true'                 
{
  "test" : {
    "foo" : {
      "properties" : {
        "field" : {
          "dynamic" : "true",
          "properties" : {
            "nested" : {
              "type" : "long"
            }
          }
        }
      }
    }
  }
}

The error that appears on the server side is:

[2011-03-19 02:32:49,171][INFO ][node                     ] [Astaroth / Asteroth] {elasticsearch/0.15.2}[25417]: started
[2011-03-19 02:32:53,375][INFO ][cluster.metadata         ] [Astaroth / Asteroth] [test] creating index, cause [auto(index api)], shards [5]/[1], mappings []
[2011-03-19 02:32:54,131][INFO ][cluster.metadata         ] [Astaroth / Asteroth] [test] created and added to cluster_state
[2011-03-19 02:32:54,491][INFO ][cluster.metadata         ] [Astaroth / Asteroth] [test] update_mapping [foo] (dynamic)
[2011-03-19 02:32:56,846][DEBUG][action.index             ] [Astaroth / Asteroth] [test][2], node[wLHdQbsoQS-eSkJNX-CLvw], [P], s[STARTED]: Failed to execute [index {[test][foo][JbZaKPbdTBa2k38UkXf7wA], source[{"field": "omg"}]}]
java.lang.NullPointerException
    at org.elasticsearch.index.mapper.xcontent.ObjectMapper.parse(ObjectMapper.java:308)
    at org.elasticsearch.index.mapper.xcontent.XContentDocumentMapper.parse(XContentDocumentMapper.java:429)
    at org.elasticsearch.index.mapper.xcontent.XContentDocumentMapper.parse(XContentDocumentMapper.java:363)
    at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:250)
    at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:187)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:418)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.access$100(TransportShardReplicationOperationAction.java:233)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:331)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)

The bug comes from the fact that the ObjectMapper.parse doesn't handle end of file here. There are two possible behaviors to adopt here:

1. Accept EOF here and Do The Right Thing.
2. Reject EOF with an error because the existing mapping expects another form of data.

In my case I want behavior 1. but I'm fairly new to ES so I don't know if this should be universal or not. Either way, the NPE is clearly a bug here.

[tsuna]

The pull request #796 implements behavior 1 and fixes the problem for me:

$ curl -XPOST 'localhost:9200/test/foo' -d '{"field": {"nested": 42}}'
{"ok":true,"_index":"test","_type":"foo","_id":"5DrNKOu0QuizJoyom8A4Tg","_version":1}
$ curl -XPOST 'localhost:9200/test/foo' -d '{"field": "omg"}'         
{"ok":true,"_index":"test","_type":"foo","_id":"XkDiUy10TNeSKRfxF827ZA","_version":1}

[kimchy]

Behavior 1 is problematic when it comes to elasticsearch, since data you expect to be indexed will not be indexed. Will push implementation for 2.

[kimchy]

NPE when the JSON to index doesn't match the mapping's expectations, closed by fb7fbc8.