Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Internal: Wait until engine has started up when acquiring searcher #7455

Closed
spinscale opened this issue Aug 26, 2014 · 0 comments · Fixed by #7456
Closed

Internal: Wait until engine has started up when acquiring searcher #7455

spinscale opened this issue Aug 26, 2014 · 0 comments · Fixed by #7456
Assignees
@s1monw
Labels
>bug v1.3.3 v1.4.0.Beta1 v2.0.0-beta1

Comments

@spinscale
Copy link
Contributor

This exception happened after a node restart and a delete-by-query hits the node immediately. Happened on 1.3.1

[2014-08-25 16:34:50,926][ERROR][index.engine.internal    ] [node_name] [2013_09][3] failed to acquire searcher, source delete_by_query
java.lang.NullPointerException
    at org.elasticsearch.index.engine.internal.InternalEngine.acquireSearcher(InternalEngine.java:694)
    at org.elasticsearch.index.shard.service.InternalIndexShard.acquireSearcher(InternalIndexShard.java:653)
    at org.elasticsearch.action.deletebyquery.TransportShardDeleteByQueryAction.shardOperationOnReplica(TransportShardDeleteByQueryAction.java:139)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$ReplicaOperationTransportHandler.messageReceived(TransportShardReplicationOperationAction.java:242)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$ReplicaOperationTransportHandler.messageReceived(TransportShardReplicationOperationAction.java:221)
    at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
[2014-08-25 16:34:50,927][WARN ][index.engine.internal    ] [node_name] [2013_09][3] failed engine [deleteByQuery/shard failed on replica]
[2014-08-25 16:35:13,744][WARN ][cluster.action.shard     ] [node_name] [2013_09][3] sending failed shard for [2013_09][3], node[0Y7oOI64Qea6GCaSh3OtLw], [R], s[INITIALIZING], indexUUID [_na_], reason [engine failure, message [deleteByQuery/shard failed on replica][EngineException[[2013_09][3] failed to acquire searcher, source delete_by_query]; nested: NullPointerException; ]]
@spinscale spinscale added the bug label Aug 26, 2014
@s1monw s1monw self-assigned this Aug 26, 2014
@s1monw s1monw mentioned this issue Aug 26, 2014
Merged
s1monw added a commit to s1monw/elasticsearch that referenced this issue Aug 26, 2014
@s1monw
[ENGINE] Wait until engine is started up when acquireing searcher
0676869 
Today we have a small window where a searcher can be acquired but the
engine is in the state of starting up. This causes a NPE triggering a
shard failure if we are fast enough. This commit fixes this situation
gracefully.

Closes elastic#7455
s1monw added a commit that referenced this issue Aug 26, 2014
@s1monw
[ENGINE] Wait until engine is started up when acquireing searcher
9753ab0 
Today we have a small window where a searcher can be acquired but the
engine is in the state of starting up. This causes a NPE triggering a
shard failure if we are fast enough. This commit fixes this situation
gracefully.

Closes #7455
s1monw added a commit that referenced this issue Aug 26, 2014
@s1monw
[ENGINE] Wait until engine is started up when acquireing searcher
77f1032 
Today we have a small window where a searcher can be acquired but the
engine is in the state of starting up. This causes a NPE triggering a
shard failure if we are fast enough. This commit fixes this situation
gracefully.

Closes #7455
@spinscale spinscale changed the title Node startup: NPE triggers failed shard Engine: Wait until engine is started up when acquiring searcher Aug 27, 2014
@clintongormley clintongormley changed the title Engine: Wait until engine is started up when acquiring searcher Internal: Wait until engine has started up when acquiring searcher Sep 8, 2014
s1monw added a commit that referenced this issue Sep 8, 2014
@s1monw @areek
[ENGINE] Wait until engine is started up when acquireing searcher
48249d9 
Today we have a small window where a searcher can be acquired but the
engine is in the state of starting up. This causes a NPE triggering a
shard failure if we are fast enough. This commit fixes this situation
gracefully.

Closes #7455
@s1monw s1monw mentioned this issue Oct 22, 2014
Closed
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
@s1monw
[ENGINE] Wait until engine is started up when acquireing searcher
0296da6 
Today we have a small window where a searcher can be acquired but the
engine is in the state of starting up. This causes a NPE triggering a
shard failure if we are fast enough. This commit fixes this situation
gracefully.

Closes elastic#7455
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@s1monw s1monw

Labels
>bug v1.3.3 v1.4.0.Beta1 v2.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Wait until engine is started up when acquiring searcher s1monw/elasticsearch
2 participants
@spinscale @s1monw