Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wait for completion for Enrich policy execution #47886

Merged
merged 6 commits into from Oct 14, 2019

Conversation

jbaiera
Copy link
Member

@jbaiera jbaiera commented Oct 10, 2019

This PR adds the ability to run the enrich policy execution task in the background, returning a task id instead of waiting for the completed operation.

@jbaiera jbaiera added >non-issue :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP labels Oct 10, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Ingest)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left two small comments. LGTM otherwise.

Comment on lines 238 to 241
GetTaskRequest getPolicyTaskRequest = new GetTaskRequest().setTaskId(executeResponse.getTaskId()).setWaitForCompletion(true);
GetTaskResponse taskResponse = client().execute(GetTaskAction.INSTANCE, getPolicyTaskRequest).actionGet();
assertThat(((ExecuteEnrichPolicyStatus) taskResponse.getTask().getTask().getStatus()).getPhase(),
is(ExecuteEnrichPolicyStatus.PolicyPhases.COMPLETE));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would place this inside an assertBusy(...), because there is no guarantee that the task is completed (especially on slow build machines).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the assertBusy line. In the code the task request has wait_for_completion set to true, is it possible that this call can come back before the task is completed even on slow hardware? I suppose at the very least assertBusy insulates the test against timeout failures

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in this test wait_for_completion is set to false, right? So it is likely that it returns and then the task isn't completed?

If wait_for_completion is true (which is the default) then the api doesn't return until the force merge has completed and therefor the associated task has been completed. So in this case it shouldn't be possible for the task not being in a completed state.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nevermind my comment... I missed that wait_for_completion is also set on GetTaskRequest. Then the task api will not return until the task in question has been completed. I didn't know this.

So the assertBusy(...) doesn't make sense and can be removed.

@jbaiera jbaiera merged commit b0ccce2 into elastic:enrich Oct 14, 2019
@jbaiera jbaiera deleted the enrich-wait-for-completion branch October 14, 2019 19:38
jbaiera added a commit that referenced this pull request Oct 14, 2019
This PR adds the ability to run the enrich policy execution task in the background,
returning a task id instead of waiting for the completed operation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >non-issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants