Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ML] Get all jobs stats request can potentially generate lots of cluster traffic and cause OOM #33515

Closed
dimitris-athanasiou opened this issue Sep 7, 2018 · 1 comment
Closed

[ML] Get all jobs stats request can potentially generate lots of cluster traffic and cause OOM #33515

dimitris-athanasiou opened this issue Sep 7, 2018 · 1 comment
Assignees
@dimitris-athanasiou
Labels
>bug :ml Machine learning

Comments

@dimitris-athanasiou
Copy link
Contributor

When calling the get-jobs-stats action with _all the tasks are not filtered properly. All ES tasks match resulting in loads of cluster traffic and the accumulation of the individual task responses can add up to a lot of memory. This is a hard issue to reproduce.
@dimitris-athanasiou dimitris-athanasiou added >bug :ml Machine learning labels Sep 7, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core

@dimitris-athanasiou dimitris-athanasiou self-assigned this Sep 7, 2018
@dimitris-athanasiou dimitris-athanasiou mentioned this issue Sep 7, 2018
Merged
dimitris-athanasiou added a commit that referenced this issue Sep 9, 2018
@dimitris-athanasiou
[ML] Get job stats request should filter non-ML job tasks (#33516)
fcb15b0 
When requesting job stats for `_all`, all ES tasks are accepted
resulting to loads of cluster traffic and a memory overhead.
This commit correctly filters out non ML job tasks.

Closes #33515
dimitris-athanasiou added a commit that referenced this issue Sep 9, 2018
@dimitris-athanasiou
[ML] Get job stats request should filter non-ML job tasks (#33516)
ca49faa 
When requesting job stats for `_all`, all ES tasks are accepted
resulting to loads of cluster traffic and a memory overhead.
This commit correctly filters out non ML job tasks.

Closes #33515
dimitris-athanasiou added a commit that referenced this issue Sep 9, 2018
@dimitris-athanasiou
[ML] Get job stats request should filter non-ML job tasks (#33516)
1e9df9b 
When requesting job stats for `_all`, all ES tasks are accepted
resulting to loads of cluster traffic and a memory overhead.
This commit correctly filters out non ML job tasks.

Closes #33515
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dimitris-athanasiou dimitris-athanasiou

Labels
>bug :ml Machine learning
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dimitris-athanasiou @elasticmachine