Execute EnrichPolicyRunner on a non dedicated master node. #76881
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
martijnvg
force-pushed
the
enrich_delegate_policy_runner_to_non_dedicated_master_node
branch
2 times, most recently
from
9b45d01
to
dd73f02
Compare
Introduce an internal action that the execute policy action delegates to. This to ensure that the actual policy execution is never executed on the elected master node or dedicated master nodes. In case the cluster consists out of a single node then the internal action will attempt to execute on the current/local node. The actual enrich policy execution is encapsulated in the `EnrichPolicyRunner` class. This class manages the execution of several API calls, so this itself isn't doing anything heavy. However the coordination of these api calls (in particular the reindex api call) may involve some non-neglectable work/overhead and this shouldn't be performed on the elected master or any other dedicated master node. Closes elastic#70436
martijnvg
force-pushed
the
enrich_delegate_policy_runner_to_non_dedicated_master_node
branch
from
dd73f02
to
5d7a948
Compare
martijnvg
added
:Data Management/Ingest Node
elasticmachine
added
the
Team:Data Management
|
Pinging @elastic/es-core-features (Team:Core/Features) |
danhermann
approved these changes
Aug 25, 2021
.../enrich/src/main/java/org/elasticsearch/xpack/enrich/action/InternalExecutePolicyAction.java
Outdated
Show resolved
Hide resolved
jbaiera
reviewed
Aug 30, 2021
.../enrich/src/main/java/org/elasticsearch/xpack/enrich/action/InternalExecutePolicyAction.java
Outdated
Show resolved
Hide resolved
.../enrich/src/main/java/org/elasticsearch/xpack/enrich/action/InternalExecutePolicyAction.java
Show resolved
Hide resolved
.../enrich/src/main/java/org/elasticsearch/xpack/enrich/action/InternalExecutePolicyAction.java
Outdated
Show resolved
Hide resolved
.../enrich/src/main/java/org/elasticsearch/xpack/enrich/action/InternalExecutePolicyAction.java
Show resolved
Hide resolved
…runner_to_non_dedicated_master_node
… response but not release the policy immediately. Execute get task request with wait_for_completion=true and when that returns release the policy.
…runner_to_non_dedicated_master_node
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Sep 2, 2021
Backporting elastic#76881 to 7.x branch. Introduce an internal action that the execute policy action delegates to. This to ensure that the actual policy execution is never executed on the elected master node or dedicated master nodes. In case the cluster consists out of a single node then the internal action will attempt to execute on the current/local node. The actual enrich policy execution is encapsulated in the `EnrichPolicyRunner` class. This class manages the execution of several API calls, so this itself isn't doing anything heavy. However the coordination of these api calls (in particular the reindex api call) may involve some non-neglectable work/overhead and this shouldn't be performed on the elected master or any other dedicated master node. Closes elastic#70436
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Sep 2, 2021
Backporting elastic#76881 to 7.x branch. Introduce an internal action that the execute policy action delegates to. This to ensure that the actual policy execution is never executed on the elected master node or dedicated master nodes. In case the cluster consists out of a single node then the internal action will attempt to execute on the current/local node. The actual enrich policy execution is encapsulated in the `EnrichPolicyRunner` class. This class manages the execution of several API calls, so this itself isn't doing anything heavy. However the coordination of these api calls (in particular the reindex api call) may involve some non-neglectable work/overhead and this shouldn't be performed on the elected master or any other dedicated master node. Closes elastic#70436
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Sep 2, 2021
Backporting elastic#76881 to 7.15 branch. Introduce an internal action that the execute policy action delegates to. This to ensure that the actual policy execution is never executed on the elected master node or dedicated master nodes. In case the cluster consists out of a single node then the internal action will attempt to execute on the current/local node. The actual enrich policy execution is encapsulated in the `EnrichPolicyRunner` class. This class manages the execution of several API calls, so this itself isn't doing anything heavy. However the coordination of these api calls (in particular the reindex api call) may involve some non-neglectable work/overhead and this shouldn't be performed on the elected master or any other dedicated master node. Closes elastic#70436
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Sep 2, 2021
martijnvg
added a commit
that referenced
this pull request
Sep 2, 2021
Backporting #76881 to 7.15 branch. Introduce an internal action that the execute policy action delegates to. This to ensure that the actual policy execution is never executed on the elected master node or dedicated master nodes. In case the cluster consists out of a single node then the internal action will attempt to execute on the current/local node. The actual enrich policy execution is encapsulated in the `EnrichPolicyRunner` class. This class manages the execution of several API calls, so this itself isn't doing anything heavy. However the coordination of these api calls (in particular the reindex api call) may involve some non-neglectable work/overhead and this shouldn't be performed on the elected master or any other dedicated master node. Closes #70436
martijnvg
added a commit
that referenced
this pull request
Sep 2, 2021
Backporting #76881 to 7.x branch. Introduce an internal action that the execute policy action delegates to. This to ensure that the actual policy execution is never executed on the elected master node or dedicated master nodes. In case the cluster consists out of a single node then the internal action will attempt to execute on the current/local node. The actual enrich policy execution is encapsulated in the `EnrichPolicyRunner` class. This class manages the execution of several API calls, so this itself isn't doing anything heavy. However the coordination of these api calls (in particular the reindex api call) may involve some non-neglectable work/overhead and this shouldn't be performed on the elected master or any other dedicated master node. Closes #70436
martijnvg
added a commit
that referenced
this pull request
Sep 2, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In case a cluster consists out of multiple nodes then delegate the execution of EnrichPolicyRunner
to a node that is not the elected master and nodes that are not dedicated master.
Closes #70436