Allow rejection of bulk items before processing #26434
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds support for bulk items to be rejected (i.e. failed) before they are processed by the TransportShardBulkAction. This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
tvernum
added
:Core/Infra/Core
bleskes
reviewed
Aug 30, 2017
| @@ -120,8 +119,10 @@ protected boolean resolveIndex() { | |||
| final IndexMetaData metaData = primary.indexSettings().getIndexMetaData(); | |||
| Translog.Location location = null; | |||
| for (int requestIndex = 0; requestIndex < request.items().length; requestIndex++) { | |||
| location = executeBulkItemRequest(metaData, primary, request, location, requestIndex, | |||
| if (request.items()[requestIndex].getPrimaryResponse() == null) { | |||
There was a problem hiding this comment.
this is tricky - see here, which shows we may have a primary response, but still execute the item... I'm not sure it's wrong, I saying "watch out" ...
Also reverts BWC checks to production-ready state
bleskes
approved these changes
Aug 31, 2017
| */ | ||
| public void abort(String index, Exception cause) { | ||
| if (primaryResponse != null) { | ||
| throw new IllegalStateException("Item already has a response (status=" + primaryResponse.status() + ")"); |
There was a problem hiding this comment.
shall we make this an assertion so the node dies?
| } | ||
| BulkShardRequest bulkShardRequest = new BulkShardRequest(shardId, RefreshPolicy.NONE, items); | ||
|
|
||
| // Preemptively reject one of the bulk items, but allow the others to proceed |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Aug 31, 2017
* master: Allow abort of bulk items before processing (elastic#26434) [Tests] Improve testing of FieldSortBuilder (elastic#26437) Upgrade to lucene-7.0.0-snapshot-d94a5f0. (elastic#26441) Implement adaptive replica selection (elastic#26128) Build: Quiet bwc build output (elastic#26430) Migrate Search requests to use Writeable reading strategies (elastic#26428) Changed version from 7.0.0-alpha1 to 6.1.0 in the nested sorting serialization check. Remove dead path conf BWC code in build
jasontedor
added a commit
that referenced
this pull request
Aug 31, 2017
In some cases a request can already be aborted and retried. This means the condition that aborting a request should only happen when an item has not been processed yet is too strict. This commit allows for a double abort. If we attempt to abort an operation that was previously processed but not aborted, we treat that as a hard failure. Relates #26434
jasontedor
pushed a commit
that referenced
this pull request
Aug 31, 2017
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction. This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
jasontedor
added a commit
that referenced
this pull request
Aug 31, 2017
In some cases a request can already be aborted and retried. This means the condition that aborting a request should only happen when an item has not been processed yet is too strict. This commit allows for a double abort. If we attempt to abort an operation that was previously processed but not aborted, we treat that as a hard failure. Relates #26434
jasontedor
pushed a commit
that referenced
this pull request
Aug 31, 2017
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction. This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
jasontedor
added a commit
that referenced
this pull request
Aug 31, 2017
In some cases a request can already be aborted and retried. This means the condition that aborting a request should only happen when an item has not been processed yet is too strict. This commit allows for a double abort. If we attempt to abort an operation that was previously processed but not aborted, we treat that as a hard failure. Relates #26434
jasontedor
pushed a commit
that referenced
this pull request
Aug 31, 2017
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction. This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
jasontedor
added a commit
that referenced
this pull request
Aug 31, 2017
In some cases a request can already be aborted and retried. This means the condition that aborting a request should only happen when an item has not been processed yet is too strict. This commit allows for a double abort. If we attempt to abort an operation that was previously processed but not aborted, we treat that as a hard failure. Relates #26434
jasontedor
pushed a commit
that referenced
this pull request
Aug 31, 2017
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction. This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
jasontedor
added a commit
that referenced
this pull request
Aug 31, 2017
In some cases a request can already be aborted and retried. This means the condition that aborting a request should only happen when an item has not been processed yet is too strict. This commit allows for a double abort. If we attempt to abort an operation that was previously processed but not aborted, we treat that as a hard failure. Relates #26434
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds support for bulk items to be rejected (i.e. failed) before they are processed by the TransportShardBulkAction.
This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).