New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix concurrent search and index delete #42621
Changes from 1 commit
5f86ea1
8edda7b
ac02fc0
49dfb3c
d7aafcd
5fe9892
b7a24f4
4a48fb8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -44,6 +44,7 @@ | |
import org.elasticsearch.common.util.concurrent.ConcurrentMapLong; | ||
import org.elasticsearch.core.internal.io.IOUtils; | ||
import org.elasticsearch.index.Index; | ||
import org.elasticsearch.index.IndexNotFoundException; | ||
import org.elasticsearch.index.IndexService; | ||
import org.elasticsearch.index.IndexSettings; | ||
import org.elasticsearch.index.engine.Engine; | ||
|
@@ -550,12 +551,19 @@ final SearchContext createAndPutContext(ShardSearchRequest request) throws IOExc | |
SearchContext context = createContext(request); | ||
boolean success = false; | ||
try { | ||
putContext(context); | ||
if (request.scroll() != null) { | ||
openScrollContexts.incrementAndGet(); | ||
context.indexShard().getSearchOperationListener().onNewScrollContext(context); | ||
} | ||
context.indexShard().getSearchOperationListener().onNewContext(context); | ||
putContext(context); | ||
// ensure that if index is deleted concurrently, we free the context immediately, either here or in afterIndexRemoved | ||
try { | ||
indicesService.indexServiceSafe(request.shardId().getIndex()); | ||
} catch (IndexNotFoundException e) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think you need the catch clause here - we free below There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How did I miss that, thanks! Fixed in 8edda7b There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. but is this necessary or can we just rearrange the registration? |
||
freeContext(context.id()); | ||
throw e; | ||
} | ||
success = true; | ||
return context; | ||
} finally { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand why you moved the
putContext(context)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By registering the search context in
activeContexts
after having invokedonNewContext/onNewScrollContext
, we guarantee that for a specificSearchContext
, the call toonNewXXX
happens before the matching call theonFreeXXX
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see I didn't think about
SearchService.afterIndexRemoved
. my issue here is that we have to callonFreeXXX
but if one of theonNewContext /onNewScrollContext
fails we don't register and fail? I think we need extra protection for this?