Snapshot Restoration Causes a NullPointerException on Indices that Include a Null Timestamp Default

[chrisguitarguy]

I have a field with a null default _timestamp value and when I try to restore from a snapshot I get a server error caused by a NullPointerException.

Looks like the default timestamp value is not checked to see if it's null: https://github.com/elasticsearch/elasticsearch/blob/v1.4.2/src/main/java/org/elasticsearch/cluster/metadata/MappingMetaData.java#L358

I had another issue with a similar thing (#9204), maybe there's other timestamp related bugs floating around?

Stack trace:

[2015-01-26 14:55:09,604][WARN ][snapshots                ] [Gamora] [example][snap1] failed to restore snapshot
java.lang.NullPointerException
    at org.elasticsearch.cluster.metadata.MappingMetaData.initMappers(MappingMetaData.java:358)
    at org.elasticsearch.cluster.metadata.MappingMetaData.<init>(MappingMetaData.java:307)
    at org.elasticsearch.cluster.metadata.IndexMetaData$Builder.fromXContent(IndexMetaData.java:666)
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotMetaData(BlobStoreRepository.java:451)
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotMetaData(BlobStoreRepository.java:392)
    at org.elasticsearch.snapshots.RestoreService.restoreSnapshot(RestoreService.java:122)
    at org.elasticsearch.action.admin.cluster.snapshots.restore.TransportRestoreSnapshotAction.masterOperation(TransportRestoreSnapshotAction.java:78)
    at org.elasticsearch.action.admin.cluster.snapshots.restore.TransportRestoreSnapshotAction.masterOperation(TransportRestoreSnapshotAction.java:41)
    at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.run(TransportMasterNodeOperationAction.java:134)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

To reproduce:

# sample index
$curl -XPOST localhost:9200/twitter -d '{
>     "mappings": {
>         "tweet": {
>             "_timestamp": {
>                 "enabled": true,
>                 "default": null
>             }
>         }
>     }
> }'
# {"acknowledged":true}

# create a snapshot
$ curl -XPUT localhost:9200/_snapshot/example/snap1
# {"accepted":true}

# make sure our snapshot is done
$ curl -XGET localhost:9200/_snapshot/example/_all?pretty
{
  "snapshots" : [ {
    "snapshot" : "snap1",
    "indices" : [ "twitter" ],
    "state" : "SUCCESS",
    "start_time" : "2015-01-26T19:48:31.941Z",
    "start_time_in_millis" : 1422301711941,
    "end_time" : "2015-01-26T19:48:31.979Z",
    "end_time_in_millis" : 1422301711979,
    "duration_in_millis" : 38,
    "failures" : [ ],
    "shards" : {
      "total" : 5,
      "failed" : 0,
      "successful" : 5
    }
  } ]
}

# Close the index so we can restore it
$ curl -XPOST localhost:9200/twitter/_close
# {"acknowledged":true}

# try to restore
curl -XPOST localhost:9200/_snapshot/example/snap1/_restore
# {"error":"NullPointerException[null]","status":500}

[dadoonet]

You are hitting the same issue seen in #9204 which will be fixed in 1.4.3.
I'm afraid you have to wait for this version to be able to restore your index.

[chrisguitarguy]

I think you mean its the same issue as #9223 and be1610b?

However, does the code for the mapper now ignore null timestamp defaults and set them to "now"? be1610b#diff-f2918a753e681f8e6abdea55d12820d5R346

[dadoonet]

Hmm. You meant that instead of:

} else if (fieldName.equals("default") && fieldNode != null) {
   defaultTimestamp = fieldNode.toString();
}

we should write the following?

} else if (fieldName.equals("default") && fieldNode != null) {
   if (fieldNode == null) {
      defaultTimestamp = null;
   } else {
      defaultTimestamp = fieldNode.toString();
   }
}

[chrisguitarguy]

Yeah, pretty much. If default is the field name use its value whether is null or not. Not sure if that's correct, however.

} else if (fieldName.equals("default")) {
   defaultTimestamp = null == fieldNode ? null : fieldNode.toString();
}

[dadoonet]

Yeah exactly. Was testing that in the mean time. PR coming... Thanks for raising that.