Better handling of missing index failures

[synhershko]

We see the following during normal operation, when trying to add documents. Auto-create indexes is set to true.

[2012-10-10 14:00:11,259][INFO ][cluster.metadata ] [Iron Man 2020] [el-2011-11-21-0000] creating index, cause [auto(index api)], shards [1]/[0], mappings [thread]
[2012-10-10 14:00:11,294][DEBUG][action.admin.indices.stats] [Iron Man 2020] [el-2011-11-21-0000][0], node[LxU0GSjRQtmZM0h8vBxuBg], [P], s[INITIALIZING]: Failed to execute [org.elasticsearch.action.admin.indices.stats.IndicesStatsRequest@41201158]
org.elasticsearch.transport.RemoteTransportException: [Crimson Cowl][inet[/192.168.1.11:9301]][indices/stats/s]
Caused by: org.elasticsearch.indices.IndexMissingException: [el-2011-11-21-0000] missing
at org.elasticsearch.indices.InternalIndicesService.indexServiceSafe(InternalIndicesService.java:244)
at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:144)
at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:53)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$ShardTransportHandler.messageReceived(TransportBroadcastOperationAction.java:398)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$ShardTransportHandler.messageReceived(TransportBroadcastOperationAction.java:384)
at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
[2012-10-10 14:00:11,295][DEBUG][action.admin.indices.status] [Iron Man 2020] [el-2011-11-21-0000][0], node[LxU0GSjRQtmZM0h8vBxuBg], [P], s[INITIALIZING]: Failed to execute [org.elasticsearch.action.admin.indices.status.IndicesStatusRequest@1b3f459d]
org.elasticsearch.transport.RemoteTransportException: [Crimson Cowl][inet[/192.168.1.11:9301]][indices/status/s]
Caused by: org.elasticsearch.indices.IndexMissingException: [el-2011-11-21-0000] missing
at org.elasticsearch.indices.InternalIndicesService.indexServiceSafe(InternalIndicesService.java:244)
at org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:152)
at org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:59)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$ShardTransportHandler.messageReceived(TransportBroadcastOperationAction.java:398)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$ShardTransportHandler.messageReceived(TransportBroadcastOperationAction.java:384)
at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

[martijnvg]

It seems that you are executing /_stats (indices stats) requests. The auto index creation is only applicable for the index api. In the upcoming 0.20.0 version there will be the ignore_indices (#2209) request param that will help you prevent these errors.

[synhershko]

I might have had the ES Paramedic thing on, yes, but how could this have triggered this alert? I didn't delete any index, so I can't see how there could be a request for a non-existing index.

[javanna]

@synhershko Are you still experiencing this problem? Did the ignore_indices parameter help perhaps?

[synhershko]

Maybe, haven't experienced this since. Closing.