Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[APM] Service maps - Fix missing ML status for services with jobs but no anomalies #68486

Merged
merged 7 commits into from
Jun 10, 2020
Merged

[APM] Service maps - Fix missing ML status for services with jobs but no anomalies #68486

merged 7 commits into from
Jun 10, 2020

Conversation

ogupte
Copy link
Contributor

@ogupte ogupte commented Jun 8, 2020 •
edited
Loading

Closes #68485 by:

  • use the ML plugin to query for all APM jobs
  • inspect the ml job groups to find all jobs related to a particular service
  • use the mlAnomalySearch client to get ml job buckets with the max anomaly score & actual/median values for the ML description
  • return the relevant ML job with the max anomaly score for a service
  • indicate to the user that no anomalies were found for a service with an ml job
    Screen Shot 2020-06-09 at 9 12 49 AM

@ogupte
Closes elastic#68485 by:
acc3db6 
  - use the ML plugin to query for all APM jobs
  - inspect the ml job groups to find all jobs related to a particular service
  - use the mlAnomalySearch client to get ml job buckets with the max anomaly score
  - query for the model_plot buckets to obtain actual/median values for the ML description
  - return the relevant ML job with the max anomaly score for a service
  - indicate to the user that no anomalies were found for a service with an ml job
@ogupte ogupte added release_note:skip Skip the PR/issue when compiling release notes v7.9.0 v7.8 labels Jun 8, 2020
@ogupte ogupte requested review from a team as code owners June 8, 2020 03:56
@botelastic botelastic bot added the Team:APM All issues that need APM UI Team support label Jun 8, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:apm)

formgeist
formgeist reviewed Jun 8, 2020
View reviewed changes
Copy link
Contributor

@formgeist formgeist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a suggestion on the empty state copy

x-pack/plugins/apm/public/components/app/ServiceMap/Popover/Contents.tsx Outdated Show resolved Hide resolved
@ogupte
- Use the anomalyDetectorsProvider jobs API rather than the search en…
08b94cf 
…dpoint directly

- Defines a specific return type for the ml jobs api
- Update the empty anomaly data message
@ogupte ogupte requested a review from jgowdyelastic June 8, 2020 16:10
jgowdyelastic
jgowdyelastic approved these changes Jun 8, 2020
View reviewed changes
Copy link
Member

@jgowdyelastic jgowdyelastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ML changes LGTM

@ogupte
Code and types cleanup
bd1ca55
@ogupte
Return to using record result type on anomaly queries. These are the
106d0f2 
same values used in the anomaly explorer and it includes actual &
typical values which greatly improve performance of the previous query.
@ogupte
- If anomaly data is missing show a gray border around node
143c0a8
smith
smith approved these changes Jun 9, 2020
View reviewed changes
Copy link
Contributor

@smith smith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits in the comments, but works well for me. Thanks!

x-pack/plugins/apm/public/components/app/ServiceMap/Popover/Contents.tsx Outdated Show resolved Hide resolved
x-pack/plugins/apm/public/components/app/ServiceMap/cytoscapeOptions.ts Outdated Show resolved Hide resolved
x-pack/plugins/apm/public/components/app/ServiceMap/Popover/Contents.tsx Outdated Show resolved Hide resolved
@ogupte
Copy link
Contributor Author

ogupte commented Jun 9, 2020

@elasticmachine merge upstream

@rayafratkina rayafratkina added v7.8.0 and removed v7.8 labels Jun 9, 2020
@ogupte
Copy link
Contributor Author

ogupte commented Jun 9, 2020

retest

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@ogupte ogupte merged commit cac9018 into elastic:master Jun 10, 2020
@ogupte ogupte mentioned this pull request Jun 10, 2020
Merged
ogupte added a commit to ogupte/kibana that referenced this pull request Jun 10, 2020
@ogupte @elasticmachine
[APM] Service maps - Fix missing ML status for services with jobs but…
56a37be 
… no anomalies (elastic#68486)

* Closes elastic#68485 by:
  - use the ML plugin to query for all APM jobs
  - inspect the ml job groups to find all jobs related to a particular service
  - use the mlAnomalySearch client to get ml job buckets with the max anomaly score
  - query for the model_plot buckets to obtain actual/median values for the ML description
  - return the relevant ML job with the max anomaly score for a service
  - indicate to the user that no anomalies were found for a service with an ml job

* - Use the anomalyDetectorsProvider jobs API rather than the search endpoint directly
- Defines a specific return type for the ml jobs api
- Update the empty anomaly data message

* Code and types cleanup

* Return to using record result type on anomaly queries. These are the
same values used in the anomaly explorer and it includes actual &
typical values which greatly improve performance of the previous query.

* - If anomaly data is missing show a gray border around node

* - moved AnomalyDetection out of service map Contents into own component

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
ogupte added a commit to ogupte/kibana that referenced this pull request Jun 10, 2020
@ogupte
[APM] Service maps - Fix missing ML status for services with jobs but…
23f75d0 
… no anomalies (elastic#68486)

* Closes elastic#68485 by:
  - use the ML plugin to query for all APM jobs
  - inspect the ml job groups to find all jobs related to a particular service
  - use the mlAnomalySearch client to get ml job buckets with the max anomaly score
  - query for the model_plot buckets to obtain actual/median values for the ML description
  - return the relevant ML job with the max anomaly score for a service
  - indicate to the user that no anomalies were found for a service with an ml job

* - Use the anomalyDetectorsProvider jobs API rather than the search endpoint directly
- Defines a specific return type for the ml jobs api
- Update the empty anomaly data message

* Code and types cleanup

* Return to using record result type on anomaly queries. These are the
same values used in the anomaly explorer and it includes actual &
typical values which greatly improve performance of the previous query.

* - If anomaly data is missing show a gray border around node

* - moved AnomalyDetection out of service map Contents into own component

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
# Conflicts:
#	x-pack/plugins/ml/server/shared.ts
@ogupte ogupte mentioned this pull request Jun 10, 2020
Merged
ogupte added a commit that referenced this pull request Jun 10, 2020
@ogupte @elasticmachine
[APM] Service maps - Fix missing ML status for services with jobs but…
8d212ea 
… no anomalies (#68486) (#68734)

* Closes #68485 by:
  - use the ML plugin to query for all APM jobs
  - inspect the ml job groups to find all jobs related to a particular service
  - use the mlAnomalySearch client to get ml job buckets with the max anomaly score
  - query for the model_plot buckets to obtain actual/median values for the ML description
  - return the relevant ML job with the max anomaly score for a service
  - indicate to the user that no anomalies were found for a service with an ml job

* - Use the anomalyDetectorsProvider jobs API rather than the search endpoint directly
- Defines a specific return type for the ml jobs api
- Update the empty anomaly data message

* Code and types cleanup

* Return to using record result type on anomaly queries. These are the
same values used in the anomaly explorer and it includes actual &
typical values which greatly improve performance of the previous query.

* - If anomaly data is missing show a gray border around node

* - moved AnomalyDetection out of service map Contents into own component

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jun 10, 2020
@gmmorris
Merge branch 'master' into alerting/consumer-based-rbac
86f7d73 
* master: (22 commits)
  Partial revert of "Sync Kerberos + Anonymous access tests with the latest `security/_authenticate` API (user roles now include roles of anonymous user)." (elastic#68624)
  adapt some snapshot test (elastic#68489)
  [APM] Service maps - Fix missing ML status for services with jobs but no anomalies (elastic#68486)
  [skip test] apis Kerberos security Kerberos authentication finishing SPNEGO should properly set cookie and authenticate user
  [SIEM][Exceptions] - ExceptionsViewer UI component part 2 (elastic#68294)
  Surface data streams in Index Management. (elastic#67806)
  Fix edit datasource not working following changes in elastic#67234 (elastic#68583)
  [Logs + Metrics UI] Clean up async plugin initialization (elastic#67654)
  APM Storybook fixes (elastic#68671)
  Upgrade EUI to v24.1.0 (elastic#68141)
  [ML] DF Analytics: Creation wizard part 2 (elastic#68462)
  [Uptime] Fix race on overview page query (elastic#67843)
  Prefer using npm_execpath when spawning Yarn (elastic#68673)
  [Security] [Cases] Attach timeline to existing case (elastic#68580)
  Use Search API in Vega (elastic#68257)
  [Component templates] Table view (elastic#68031)
  [Uptime] Added relative date info in cert status column (elastic#67612)
  [Endpoint] Re-enable Functional test case for Endpoint related pages (elastic#68445)
  run page_load_metrics tests in visual regresssion jobs (elastic#68570)
  Enable exhaustive-deps; correct any lint warnings (elastic#68453)
  ...
ogupte added a commit that referenced this pull request Jun 10, 2020
@ogupte @elasticmachine
[7.8] [APM] Service maps - Fix missing ML status for services with jo…
2dc4c8b 
…bs but no anomalies (#68486) (#68735)

* [APM] Service maps - Fix missing ML status for services with jobs but no anomalies (#68486)

* Closes #68485 by:
  - use the ML plugin to query for all APM jobs
  - inspect the ml job groups to find all jobs related to a particular service
  - use the mlAnomalySearch client to get ml job buckets with the max anomaly score
  - query for the model_plot buckets to obtain actual/median values for the ML description
  - return the relevant ML job with the max anomaly score for a service
  - indicate to the user that no anomalies were found for a service with an ml job

* - Use the anomalyDetectorsProvider jobs API rather than the search endpoint directly
- Defines a specific return type for the ml jobs api
- Update the empty anomaly data message

* Code and types cleanup

* Return to using record result type on anomaly queries. These are the
same values used in the anomaly explorer and it includes actual &
typical values which greatly improve performance of the previous query.

* - If anomaly data is missing show a gray border around node

* - moved AnomalyDetection out of service map Contents into own component

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
# Conflicts:
#	x-pack/plugins/ml/server/shared.ts

* - exposes ML types to be referenced in APM
- uses existing mlSearch API rather than the mlAnomalySearch API

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sorenlouv sorenlouv sorenlouv left review comments

@formgeist formgeist formgeist left review comments

@bmorelli25 bmorelli25 bmorelli25 left review comments

@smith smith smith approved these changes

@jgowdyelastic jgowdyelastic jgowdyelastic approved these changes

Assignees
No one assigned
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:APM All issues that need APM UI Team support v7.8.0 v7.9.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[APM] Service map bug doesn't always display ML health indicators
9 participants
@ogupte @elasticmachine @kibanamachine @smith @sorenlouv @formgeist @bmorelli25 @jgowdyelastic @rayafratkina