New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot Restoration Causes a NullPointerException on Indices that Include a Null Timestamp Default #9426

Closed
chrisguitarguy opened this Issue Jan 26, 2015 · 5 comments

Comments

Projects
None yet
2 participants
@chrisguitarguy
Copy link

chrisguitarguy commented Jan 26, 2015

I have a field with a null default _timestamp value and when I try to restore from a snapshot I get a server error caused by a NullPointerException.

Looks like the default timestamp value is not checked to see if it's null: https://github.com/elasticsearch/elasticsearch/blob/v1.4.2/src/main/java/org/elasticsearch/cluster/metadata/MappingMetaData.java#L358

I had another issue with a similar thing (#9204), maybe there's other timestamp related bugs floating around?

Stack trace:

[2015-01-26 14:55:09,604][WARN ][snapshots                ] [Gamora] [example][snap1] failed to restore snapshot
java.lang.NullPointerException
    at org.elasticsearch.cluster.metadata.MappingMetaData.initMappers(MappingMetaData.java:358)
    at org.elasticsearch.cluster.metadata.MappingMetaData.<init>(MappingMetaData.java:307)
    at org.elasticsearch.cluster.metadata.IndexMetaData$Builder.fromXContent(IndexMetaData.java:666)
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotMetaData(BlobStoreRepository.java:451)
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotMetaData(BlobStoreRepository.java:392)
    at org.elasticsearch.snapshots.RestoreService.restoreSnapshot(RestoreService.java:122)
    at org.elasticsearch.action.admin.cluster.snapshots.restore.TransportRestoreSnapshotAction.masterOperation(TransportRestoreSnapshotAction.java:78)
    at org.elasticsearch.action.admin.cluster.snapshots.restore.TransportRestoreSnapshotAction.masterOperation(TransportRestoreSnapshotAction.java:41)
    at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.run(TransportMasterNodeOperationAction.java:134)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

To reproduce:

# sample index
$curl -XPOST localhost:9200/twitter -d '{
>     "mappings": {
>         "tweet": {
>             "_timestamp": {
>                 "enabled": true,
>                 "default": null
>             }
>         }
>     }
> }'
# {"acknowledged":true}

# create a snapshot
$ curl -XPUT localhost:9200/_snapshot/example/snap1
# {"accepted":true}

# make sure our snapshot is done
$ curl -XGET localhost:9200/_snapshot/example/_all?pretty
{
  "snapshots" : [ {
    "snapshot" : "snap1",
    "indices" : [ "twitter" ],
    "state" : "SUCCESS",
    "start_time" : "2015-01-26T19:48:31.941Z",
    "start_time_in_millis" : 1422301711941,
    "end_time" : "2015-01-26T19:48:31.979Z",
    "end_time_in_millis" : 1422301711979,
    "duration_in_millis" : 38,
    "failures" : [ ],
    "shards" : {
      "total" : 5,
      "failed" : 0,
      "successful" : 5
    }
  } ]
}

# Close the index so we can restore it
$ curl -XPOST localhost:9200/twitter/_close
# {"acknowledged":true}

# try to restore
curl -XPOST localhost:9200/_snapshot/example/snap1/_restore
# {"error":"NullPointerException[null]","status":500}
@dadoonet

This comment has been minimized.

Copy link
Member

dadoonet commented Jan 26, 2015

You are hitting the same issue seen in #9204 which will be fixed in 1.4.3.
I'm afraid you have to wait for this version to be able to restore your index.

@dadoonet dadoonet closed this Jan 26, 2015

@chrisguitarguy

This comment has been minimized.

Copy link

chrisguitarguy commented Jan 26, 2015

I think you mean its the same issue as #9223 and be1610b?

However, does the code for the mapper now ignore null timestamp defaults and set them to "now"? be1610b#diff-f2918a753e681f8e6abdea55d12820d5R346

@dadoonet

This comment has been minimized.

Copy link
Member

dadoonet commented Jan 26, 2015

Hmm. You meant that instead of:

} else if (fieldName.equals("default") && fieldNode != null) {
   defaultTimestamp = fieldNode.toString();
}

we should write the following?

} else if (fieldName.equals("default") && fieldNode != null) {
   if (fieldNode == null) {
      defaultTimestamp = null;
   } else {
      defaultTimestamp = fieldNode.toString();
   }
}
@chrisguitarguy

This comment has been minimized.

Copy link

chrisguitarguy commented Jan 26, 2015

Yeah, pretty much. If default is the field name use its value whether is null or not. Not sure if that's correct, however.

} else if (fieldName.equals("default")) {
   defaultTimestamp = null == fieldNode ? null : fieldNode.toString();
}
@dadoonet

This comment has been minimized.

Copy link
Member

dadoonet commented Jan 26, 2015

Yeah exactly. Was testing that in the mean time. PR coming... Thanks for raising that.

dadoonet added a commit to dadoonet/elasticsearch that referenced this issue Jan 26, 2015

Explicit _timestamp default null is set to now
When creating an index with:

```
PUT new_index
{
    "mappings": {
        "power": {
            "_timestamp" : {
                "enabled" : true,
                "default": null
            }
        }
    }
}
```

When restarting the cluster, `now` is applied instead of `null`. So index become:

```
{
    "mappings": {
        "power": {
            "_timestamp" : {
                "enabled" : true,
                "default": "now"
            }
        }
    }
}
```

This PR fix that and applies `null` when it was explicitly set.

Note that this won't happen anymore in 1.5 as `null` is not allowed anymore as a `default` value. See elastic#9104.

See also:

* elastic#7036
* elastic#9049
* elastic#9426#issuecomment-71534871

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Explicit _timestamp default null is set to now
When creating an index with:

```
PUT new_index
{
    "mappings": {
        "power": {
            "_timestamp" : {
                "enabled" : true,
                "default": null
            }
        }
    }
}
```

When restarting the cluster, `now` is applied instead of `null`. So index become:

```
{
    "mappings": {
        "power": {
            "_timestamp" : {
                "enabled" : true,
                "default": "now"
            }
        }
    }
}
```

This PR fix that and applies `null` when it was explicitly set.

Note that this won't happen anymore in 1.5 as `null` is not allowed anymore as a `default` value. See elastic#9104.

See also:

* elastic#7036
* elastic#9049
* elastic#9426#issuecomment-71534871
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment