Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Explain Log Rate Spikes: Group results API. #140683

Merged
merged 7 commits into from Sep 14, 2022

Conversation

walterra
Copy link
Contributor

@walterra walterra commented Sep 14, 2022

Summary

Part of #138117.

Extends the /internal/aiops/explain_log_rate_spikes with an option to extend the analysis and summarize significant field/value pairs into groups using the frequent_items aggregation.

Checklist

@walterra walterra self-assigned this Sep 14, 2022
@walterra walterra added :ml release_note:skip Skip the PR/issue when compiling release notes v8.5.0 Feature:ML/AIOps ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis release_note:enhancement and removed release_note:skip Skip the PR/issue when compiling release notes labels Sep 14, 2022
@walterra walterra marked this pull request as ready for review September 14, 2022 12:33
@walterra walterra requested a review from a team as a code owner September 14, 2022 12:33
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@walterra walterra changed the title [aiops] Explain Log Rate Spikes: Group results API. [ML] Explain Log Rate Spikes: Group results API. Sep 14, 2022
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/ml-agg-utils 35 39 +4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
aiops 479.7KB 479.9KB +139.0B
Unknown metric groups

API count

id before after diff
@kbn/ml-agg-utils 53 58 +5

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

// get unique fields that are left
const fields = [...new Set(terms.map((t) => t.fieldName))];

// TODO add query params
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this be added in a follow up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct!

*/

import { max } from 'd3-array';
// import { omit, uniq } from 'lodash';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can remove this commented line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave it in because it's part of another commented part around categories which we might need later.


// TODO - add handling of creating * as next level of tree

if (Object.keys(getValueCounts(filteredItemSets, nextField)).length > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance the break condition is never hit and we stay in an infinite while loop? Like could that length always be zero for some reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is approach is based on the original Python integration. I'll stick to it for now, once we have better test coverage we can maybe refactor to get rid of the while(true).

Copy link
Contributor

@alvarezmelissa87 alvarezmelissa87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and LGTM - okay to address small comments in follow ups ⚡

displayOther: boolean,
fields: string[] = []
) {
// const candidates = uniq(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: commented code

leaves: ChangePointGroup[],
level = 1
) {
// console.log(`${'-'.repeat(level)} ${tree.name} ${tree.children.length}`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can remove this comment

@walterra walterra merged commit 5ba23e4 into elastic:main Sep 14, 2022
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Sep 14, 2022
@walterra walterra deleted the ml-aiops-grouping branch September 14, 2022 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:ML/AIOps ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis :ml release_note:enhancement v8.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants