Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Creates categorization job from pattern analysis #170567

Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
fc7bb19
[ML] Create categorization job from pattern analysis
jgowdyelastic Nov 3, 2023
1425e01
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 6, 2023
7872002
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 7, 2023
00f5a00
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 7, 2023
53e10c5
adding link to wizard
jgowdyelastic Nov 7, 2023
8fc8697
fixing per partition
jgowdyelastic Nov 7, 2023
080643a
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 8, 2023
ebfe734
better validation
jgowdyelastic Nov 8, 2023
7c0cf71
fix dependency cache issue
jgowdyelastic Nov 8, 2023
f48b6c3
passing stop on warn to wizard
jgowdyelastic Nov 8, 2023
54017c1
types refactor
jgowdyelastic Nov 8, 2023
ff2f4e9
making uiActions optional
jgowdyelastic Nov 8, 2023
073000e
moving button to sepatate component
jgowdyelastic Nov 8, 2023
408d464
moving ui action to package
jgowdyelastic Nov 8, 2023
8625443
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 8, 2023
fd08c9a
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Nov 8, 2023
f99fe22
[CI] Auto-commit changed files from 'node scripts/lint_packages --fix'
kibanamachine Nov 8, 2023
f9e718f
[CI] Auto-commit changed files from 'node scripts/generate codeowners'
kibanamachine Nov 8, 2023
dacbecb
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 9, 2023
3d8b6f3
translations
jgowdyelastic Nov 9, 2023
583f366
Add machine learning icon
jgowdyelastic Nov 9, 2023
4cb5543
flyout refactor
jgowdyelastic Nov 9, 2023
bfa89a9
using correct time range for results
jgowdyelastic Nov 9, 2023
156b2df
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 9, 2023
4c97a64
private methods
jgowdyelastic Nov 9, 2023
9142c26
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 9, 2023
ce5a26d
optional dashboard
jgowdyelastic Nov 9, 2023
31b87c1
clean up
jgowdyelastic Nov 9, 2023
f5d3f76
adjusting gutter size
jgowdyelastic Nov 9, 2023
18f7f7a
Merge remote-tracking branch 'origin/main' into create-categorization…
jgowdyelastic Nov 9, 2023
be97917
fix after merge with main
jgowdyelastic Nov 9, 2023
a1eb0c8
updating trigger text
jgowdyelastic Nov 10, 2023
b6359b0
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 13, 2023
7802419
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 13, 2023
1afac7e
adding partition field to influencers
jgowdyelastic Nov 14, 2023
3fddf0b
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 16, 2023
d32017d
improving rison parsing
jgowdyelastic Nov 16, 2023
055a53b
removing question mark
jgowdyelastic Nov 16, 2023
7eaabbd
Merge branch 'main' into create-categorization-job-from-pattern-analysis
kibanamachine Nov 21, 2023
8451388
Merge remote-tracking branch 'origin/main' into create-categorization…
jgowdyelastic Nov 21, 2023
aa2cf7c
Merge branch 'create-categorization-job-from-pattern-analysis' of git…
jgowdyelastic Nov 21, 2023
791a0d6
Merge branch 'main' into create-categorization-job-from-pattern-analysis
jgowdyelastic Nov 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Expand Up @@ -533,6 +533,7 @@ x-pack/packages/ml/route_utils @elastic/ml-ui
x-pack/packages/ml/runtime_field_utils @elastic/ml-ui
x-pack/packages/ml/string_hash @elastic/ml-ui
x-pack/packages/ml/trained_models_utils @elastic/ml-ui
x-pack/packages/ml/ui_actions @elastic/ml-ui
x-pack/packages/ml/url_state @elastic/ml-ui
packages/kbn-monaco @elastic/appex-sharedux
x-pack/plugins/monitoring_collection @elastic/obs-ux-infra_services-team
Expand Down
1 change: 1 addition & 0 deletions package.json
Expand Up @@ -552,6 +552,7 @@
"@kbn/ml-runtime-field-utils": "link:x-pack/packages/ml/runtime_field_utils",
"@kbn/ml-string-hash": "link:x-pack/packages/ml/string_hash",
"@kbn/ml-trained-models-utils": "link:x-pack/packages/ml/trained_models_utils",
"@kbn/ml-ui-actions": "link:x-pack/packages/ml/ui_actions",
"@kbn/ml-url-state": "link:x-pack/packages/ml/url_state",
"@kbn/monaco": "link:packages/kbn-monaco",
"@kbn/monitoring-collection-plugin": "link:x-pack/plugins/monitoring_collection",
Expand Down
2 changes: 2 additions & 0 deletions tsconfig.base.json
Expand Up @@ -1060,6 +1060,8 @@
"@kbn/ml-string-hash/*": ["x-pack/packages/ml/string_hash/*"],
"@kbn/ml-trained-models-utils": ["x-pack/packages/ml/trained_models_utils"],
"@kbn/ml-trained-models-utils/*": ["x-pack/packages/ml/trained_models_utils/*"],
"@kbn/ml-ui-actions": ["x-pack/packages/ml/ui_actions"],
"@kbn/ml-ui-actions/*": ["x-pack/packages/ml/ui_actions/*"],
"@kbn/ml-url-state": ["x-pack/packages/ml/url_state"],
"@kbn/ml-url-state/*": ["x-pack/packages/ml/url_state/*"],
"@kbn/monaco": ["packages/kbn-monaco"],
Expand Down
Expand Up @@ -7,7 +7,7 @@

import React, { FC, useCallback, useMemo, useState } from 'react';

import { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';

import {
EuiButton,
Expand Down
Expand Up @@ -8,7 +8,7 @@
import moment from 'moment';
import type { TimefilterContract } from '@kbn/data-plugin/public';
import dateMath from '@kbn/datemath';
import { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import { i18n } from '@kbn/i18n';
import type { ToastsStart, HttpStart } from '@kbn/core/public';
import type { DataView } from '@kbn/data-views-plugin/public';
Expand Down
Expand Up @@ -6,7 +6,7 @@
*/

import type * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';

import type { HttpStart } from '@kbn/core/public';

Expand Down
Expand Up @@ -5,7 +5,7 @@
* 2.0.
*/

import { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/types';
import { cloneDeep } from 'lodash';
import { isPopulatedObject } from '@kbn/ml-is-populated-object';

Expand Down
3 changes: 3 additions & 0 deletions x-pack/packages/ml/ui_actions/README.md
@@ -0,0 +1,3 @@
# @kbn/ml-ui-actions

Empty package generated by @kbn/generate
12 changes: 12 additions & 0 deletions x-pack/packages/ml/ui_actions/index.ts
@@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

export {
CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_ACTION,
CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_TRIGGER,
type CreateCategorizationADJobContext,
} from './src/ui_actions';
12 changes: 12 additions & 0 deletions x-pack/packages/ml/ui_actions/jest.config.js
@@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

module.exports = {
preset: '@kbn/test/jest_node',
rootDir: '../../../..',
roots: ['<rootDir>/x-pack/packages/ml/ui_actions'],
};
5 changes: 5 additions & 0 deletions x-pack/packages/ml/ui_actions/kibana.jsonc
@@ -0,0 +1,5 @@
{
"type": "shared-common",
"id": "@kbn/ml-ui-actions",
"owner": "@elastic/ml-ui"
}
6 changes: 6 additions & 0 deletions x-pack/packages/ml/ui_actions/package.json
@@ -0,0 +1,6 @@
{
"name": "@kbn/ml-ui-actions",
"private": true,
"version": "1.0.0",
"license": "Elastic License 2.0"
}
22 changes: 22 additions & 0 deletions x-pack/packages/ml/ui_actions/src/ui_actions.ts
@@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import type { DataView, DataViewField } from '@kbn/data-views-plugin/common';
import type { TimeRange } from '@kbn/es-query';

export interface CreateCategorizationADJobContext {
field: DataViewField;
dataView: DataView;
query: QueryDslQueryContainer;
timeRange: TimeRange;
}

export const CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_ACTION = 'createMLADCategorizationJobAction';

export const CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_TRIGGER =
'CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_TRIGGER';
20 changes: 20 additions & 0 deletions x-pack/packages/ml/ui_actions/tsconfig.json
@@ -0,0 +1,20 @@
{
"extends": "../../../../tsconfig.base.json",
"compilerOptions": {
"outDir": "target/types",
"types": [
"jest",
"node"
]
},
"include": [
"**/*.ts",
],
"exclude": [
"target/**/*"
],
"kbn_references": [
"@kbn/data-views-plugin",
"@kbn/es-query",
]
}
@@ -0,0 +1,76 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React, { FC } from 'react';

import moment from 'moment';
import { EuiButtonEmpty } from '@elastic/eui';
import type { DataViewField, DataView } from '@kbn/data-views-plugin/common';
import type { QueryDslQueryContainer } from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import {
CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_TRIGGER,
type CreateCategorizationADJobContext,
} from '@kbn/ml-ui-actions';
import { FormattedMessage } from '@kbn/i18n-react';

import { useAiopsAppContext } from '../../hooks/use_aiops_app_context';

interface Props {
dataView: DataView;
field: DataViewField;
query: QueryDslQueryContainer;
earliest: number | undefined;
latest: number | undefined;
}

export const CreateCategorizationJobButton: FC<Props> = ({
dataView,
field,
query,
earliest,
latest,
}) => {
const {
uiActions,
application: { capabilities },
} = useAiopsAppContext();

const createADJob = () => {
if (uiActions === undefined) {
return;
}

const triggerOptions: CreateCategorizationADJobContext = {
dataView,
field,
query,
timeRange: { from: moment(earliest).toISOString(), to: moment(latest).toISOString() },
};
uiActions.getTrigger(CREATE_PATTERN_ANALYSIS_TO_ML_AD_JOB_TRIGGER).exec(triggerOptions);
};

if (uiActions === undefined || capabilities.ml.canCreateJob === false) {
return null;
}

return (
<>
<EuiButtonEmpty
data-test-subj="aiopsLogCategorizationFlyoutAdJobButton"
onClick={() => createADJob()}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you just pass in createADJob here without creating a wrapper function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in d32017d

flush="left"
iconSide="left"
iconType={'machineLearningApp'}
>
<FormattedMessage
id="xpack.aiops.categorizeFlyout.findAnomalies"
defaultMessage="Find anomalies in patterns"
/>
</EuiButtonEmpty>
</>
);
};
Expand Up @@ -44,6 +44,7 @@ import { TechnicalPreviewBadge } from './technical_preview_badge';
import { LoadingCategorization } from './loading_categorization';
import { useValidateFieldRequest } from './use_validate_category_field';
import { FieldValidationCallout } from './category_validation_callout';
import { CreateCategorizationJobButton } from './create_categorization_job';

export interface LogCategorizationPageProps {
dataView: DataView;
Expand Down Expand Up @@ -261,17 +262,21 @@ export const LogCategorizationFlyout: FC<LogCategorizationPageProps> = ({
</EuiFlexGroup>
</EuiFlyoutHeader>
<EuiFlyoutBody data-test-subj="mlJobSelectorFlyoutBody">
<CreateCategorizationJobButton
dataView={dataView}
field={selectedField}
query={searchQuery}
earliest={earliest}
latest={latest}
/>
<FieldValidationCallout validationResults={fieldValidationResult} />

{loading === true ? <LoadingCategorization onClose={onClose} /> : null}

<InformationText
loading={loading}
categoriesLength={data?.categories?.length ?? null}
eventRateLength={eventRate.length}
fieldSelected={selectedField !== null}
/>

{loading === false && data !== null && data.categories.length > 0 ? (
<CategoryTable
categories={data.categories}
Expand Down
Expand Up @@ -19,7 +19,7 @@ import { UI_SETTINGS } from '@kbn/data-plugin/public';
import { DatePickerContextProvider, type DatePickerDependencies } from '@kbn/ml-date-picker';
import { StorageContextProvider } from '@kbn/ml-local-storage';
import type { AiopsPluginStartDeps } from '../../types';
import { AiopsAppContext } from '../../hooks/use_aiops_app_context';
import { AiopsAppContext, type AiopsAppDependencies } from '../../hooks/use_aiops_app_context';
import { LogCategorizationFlyout } from './log_categorization_for_flyout';
import { AIOPS_STORAGE_KEYS } from '../../types/storage';

Expand All @@ -41,7 +41,7 @@ export async function showCategorizeFlyout(
resolve();
};

const appDependencies = {
const appDependencies: AiopsAppDependencies = {
notifications,
uiSettings,
http,
Expand Down
5 changes: 5 additions & 0 deletions x-pack/plugins/aiops/public/hooks/use_aiops_app_context.ts
Expand Up @@ -33,6 +33,7 @@ import type { PresentationUtilPluginStart } from '@kbn/presentation-util-plugin/
import type { EmbeddableStart } from '@kbn/embeddable-plugin/public';
import type { CasesUiStart } from '@kbn/cases-plugin/public';
import type { UsageCollectionSetup } from '@kbn/usage-collection-plugin/public';
import type { UiActionsStart } from '@kbn/ui-actions-plugin/public';

/**
* AIOps App Dependencies to be provided via React context.
Expand Down Expand Up @@ -97,6 +98,10 @@ export interface AiopsAppDependencies {
* Used to create lens embeddables.
*/
lens: LensPublicStart;
/**
* UI actions.
*/
uiActions?: UiActionsStart;
/**
* Internationalisation service
*/
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/aiops/tsconfig.json
Expand Up @@ -66,6 +66,7 @@
"@kbn/ml-chi2test",
"@kbn/usage-collection-plugin",
"@kbn/analytics",
"@kbn/ml-ui-actions",
],
"exclude": [
"target/**/*",
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/ml/common/constants/locator.ts
Expand Up @@ -55,6 +55,7 @@ export const ML_PAGES = {
ANOMALY_DETECTION_CREATE_JOB_SELECT_TYPE: 'jobs/new_job/step/job_type',
ANOMALY_DETECTION_CREATE_JOB_SELECT_INDEX: 'jobs/new_job/step/index_or_search',
ANOMALY_DETECTION_CREATE_JOB_FROM_LENS: 'jobs/new_job/from_lens',
ANOMALY_DETECTION_CREATE_JOB_FROM_PATTERN_ANALYSIS: 'jobs/new_job/from_pattern_analysis',
ANOMALY_DETECTION_CREATE_JOB_FROM_MAP: 'jobs/new_job/from_map',
ANOMALY_DETECTION_MODULES_VIEW_OR_CREATE: 'modules/check_view_or_create',
SETTINGS: 'settings',
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/ml/common/constants/new_job.ts
Expand Up @@ -26,6 +26,7 @@ export enum CREATED_BY_LABEL {
APM_TRANSACTION = 'ml-module-apm-transaction',
SINGLE_METRIC_FROM_LENS = 'single-metric-wizard-from-lens',
MULTI_METRIC_FROM_LENS = 'multi-metric-wizard-from-lens',
CATEGORIZATION_FROM_PATTERN_ANALYSIS = 'categorization-wizard-from-pattern-analysis',
}

export const DEFAULT_MODEL_MEMORY_LIMIT = '10MB';
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/ml/common/types/locator.ts
Expand Up @@ -49,6 +49,7 @@ export type MlGenericUrlState = MLPageState<
| typeof ML_PAGES.ANOMALY_DETECTION_CREATE_JOB_ADVANCED
| typeof ML_PAGES.ANOMALY_DETECTION_CREATE_JOB_FROM_LENS
| typeof ML_PAGES.ANOMALY_DETECTION_CREATE_JOB_FROM_MAP
| typeof ML_PAGES.ANOMALY_DETECTION_CREATE_JOB_FROM_PATTERN_ANALYSIS
| typeof ML_PAGES.ANOMALY_DETECTION_CREATE_JOB_SELECT_TYPE
| typeof ML_PAGES.ANOMALY_DETECTION_CREATE_JOB_SELECT_INDEX
| typeof ML_PAGES.DATA_FRAME_ANALYTICS_CREATE_JOB
Expand Down
Expand Up @@ -65,6 +65,7 @@ export const ChangePointDetectionPage: FC = () => {
'share',
'storage',
'theme',
'uiActions',
'uiSettings',
'unifiedSearch',
'usageCollection',
Expand Down
Expand Up @@ -56,6 +56,7 @@ export const LogCategorizationPage: FC = () => {
'share',
'storage',
'theme',
'uiActions',
'uiSettings',
'unifiedSearch',
])}
Expand Down
Expand Up @@ -59,6 +59,7 @@ export const LogRateAnalysisPage: FC = () => {
'share',
'storage',
'theme',
'uiActions',
'uiSettings',
'unifiedSearch',
])}
Expand Down
Expand Up @@ -361,7 +361,7 @@ const CategoryExamples: FC<{ definition: CategoryDefinition; examples: string[]
<EuiFlexGroup
direction="column"
justifyContent="center"
gutterSize="m"
gutterSize="xs"
className="mlAnomalyCategoryExamples"
>
{definition !== undefined && definition.terms && (
Expand Down