Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Explain Log Rate Spikes: Fix mini histograms for groups with multiples values per field. #147597

Merged
merged 6 commits into from
Dec 20, 2022

Conversation

walterra
Copy link
Contributor

@walterra walterra commented Dec 15, 2022

Summary

Part of #146162.

For groups that have multiple values for the same field, the group histogram query wasn't able to fetch data because it filters with a bool must and individual term aggregations. This PR fixes it by using a terms aggregation if there are multiple values for a field so just these get treated as OR.

Before:

image

After:

image

Checklist

@walterra walterra added bug Fixes for quality problems that affect the customer experience release_note:fix :ml Feature:ML/AIOps ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis v8.6.0 v8.7.0 labels Dec 15, 2022
@walterra walterra self-assigned this Dec 15, 2022
@walterra walterra requested a review from a team as a code owner December 15, 2022 10:08
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

Copy link
Contributor

@alvarezmelissa87 alvarezmelissa87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ⚡

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #14 / saved objects tagging - functional tests visualize integration editing allows to assign tags to an existing visualization

Metrics [docs]

Unknown metric groups

ESLint disabled in files

id before after diff
osquery 1 2 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 61 67 +6
osquery 109 115 +6
securitySolution 439 445 +6
total +20

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 70 76 +6
osquery 110 117 +7
securitySolution 516 522 +6
total +21

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

@qn895
Copy link
Member

qn895 commented Dec 20, 2022

Code LGTM

@walterra walterra merged commit 3a48b7a into elastic:main Dec 20, 2022
@walterra walterra deleted the 146162d-fix-group-mini-histograms branch December 20, 2022 15:31
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 20, 2022
…tiples values per field. (elastic#147597)

For groups that have multiple values for the same field, the group
histogram query wasn't able to fetch data because it filters with a bool
`must` and individual `term` aggregations. This PR fixes it by using a
`terms` aggregation if there are multiple values for a field so just
these get treated as `OR`.

(cherry picked from commit 3a48b7a)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.6

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Dec 20, 2022
…th multiples values per field. (#147597) (#147865)

# Backport

This will backport the following commits from `main` to `8.6`:
- [[ML] Explain Log Rate Spikes: Fix mini histograms for groups with
multiples values per field.
(#147597)](#147597)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Walter
Rafelsberger","email":"walter.rafelsberger@elastic.co"},"sourceCommit":{"committedDate":"2022-12-20T15:31:12Z","message":"[ML]
Explain Log Rate Spikes: Fix mini histograms for groups with multiples
values per field. (#147597)\n\nFor groups that have multiple values for
the same field, the group\r\nhistogram query wasn't able to fetch data
because it filters with a bool\r\n`must` and individual `term`
aggregations. This PR fixes it by using a\r\n`terms` aggregation if
there are multiple values for a field so just\r\nthese get treated as
`OR`.","sha":"3a48b7a84e587f0d5b9bffef0abecbce9c62e2e6","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix",":ml","Feature:ML/AIOps","v8.6.0","v8.7.0"],"number":147597,"url":"#147597
Explain Log Rate Spikes: Fix mini histograms for groups with multiples
values per field. (#147597)\n\nFor groups that have multiple values for
the same field, the group\r\nhistogram query wasn't able to fetch data
because it filters with a bool\r\n`must` and individual `term`
aggregations. This PR fixes it by using a\r\n`terms` aggregation if
there are multiple values for a field so just\r\nthese get treated as
`OR`.","sha":"3a48b7a84e587f0d5b9bffef0abecbce9c62e2e6"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"#147597
Explain Log Rate Spikes: Fix mini histograms for groups with multiples
values per field. (#147597)\n\nFor groups that have multiple values for
the same field, the group\r\nhistogram query wasn't able to fetch data
because it filters with a bool\r\n`must` and individual `term`
aggregations. This PR fixes it by using a\r\n`terms` aggregation if
there are multiple values for a field so just\r\nthese get treated as
`OR`.","sha":"3a48b7a84e587f0d5b9bffef0abecbce9c62e2e6"}}]}] BACKPORT-->

Co-authored-by: Walter Rafelsberger <walter.rafelsberger@elastic.co>
crespocarlos pushed a commit to crespocarlos/kibana that referenced this pull request Dec 23, 2022
…tiples values per field. (elastic#147597)

For groups that have multiple values for the same field, the group
histogram query wasn't able to fetch data because it filters with a bool
`must` and individual `term` aggregations. This PR fixes it by using a
`terms` aggregation if there are multiple values for a field so just
these get treated as `OR`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:ML/AIOps ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis :ml release_note:fix v8.6.0 v8.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants