Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Create rule from timeline #143020

Merged
merged 58 commits into from
Dec 8, 2022
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
5e2926e
start
stephmilovic Oct 10, 2022
9e27d07
fixings
stephmilovic Oct 10, 2022
22667b2
fixings
stephmilovic Oct 10, 2022
79c58c4
fix tests
stephmilovic Oct 11, 2022
8cfede7
more wip
stephmilovic Oct 11, 2022
0a73748
fix url rison
stephmilovic Oct 11, 2022
db55207
fix fields
stephmilovic Oct 11, 2022
626cb40
ok
stephmilovic Oct 11, 2022
cc3f91b
fixed
stephmilovic Oct 11, 2022
9bb7fd7
fix line
stephmilovic Oct 11, 2022
2c539df
Merge branch 'main' into on_week_timeline_rules
stephmilovic Oct 12, 2022
e16d6b2
sourcerer based approach
stephmilovic Oct 12, 2022
ffa8fa3
Merge branch 'main' into on_week_timeline_rules
stephmilovic Oct 17, 2022
cc82005
fixes
stephmilovic Oct 17, 2022
54e3414
more fixing
stephmilovic Oct 17, 2022
f805875
Merge branch 'main' into on_week_timeline_rules
stephmilovic Oct 17, 2022
d75150d
fixies
stephmilovic Oct 17, 2022
35fd82a
test fixes
stephmilovic Oct 18, 2022
32f67ec
Merge branch 'main' into on_week_timeline_rules
kibanamachine Oct 18, 2022
acdfb0d
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 17, 2022
d302c47
move files
stephmilovic Nov 17, 2022
5d989e7
naming and fix import
stephmilovic Nov 17, 2022
3a270a5
fixie
stephmilovic Nov 17, 2022
4a86f3a
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 18, 2022
4faa184
better naming and organization
stephmilovic Nov 18, 2022
83d606a
better
stephmilovic Nov 18, 2022
2adc118
more tests
stephmilovic Nov 18, 2022
324fe75
fix
stephmilovic Nov 18, 2022
d48119f
fixd
stephmilovic Nov 18, 2022
0f3f329
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 18, 2022
2694a5a
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 18, 2022
7eec14e
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 21, 2022
16cc5ae
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 22, 2022
aa31d4e
wtf
stephmilovic Nov 22, 2022
e933eb4
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 23, 2022
2ea52de
better?
stephmilovic Nov 23, 2022
52d47a6
fix
stephmilovic Nov 23, 2022
ef6cbd5
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 28, 2022
b270044
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 28, 2022
b54c15a
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 29, 2022
30202ce
fix
stephmilovic Nov 30, 2022
e4fda72
fix tests
stephmilovic Nov 30, 2022
11e8541
fix tests
stephmilovic Nov 30, 2022
00fb55c
Merge branch 'main' into on_week_timeline_rules
stephmilovic Nov 30, 2022
e1af016
add more tests
stephmilovic Dec 1, 2022
9ede9a4
Merge branch 'main' into on_week_timeline_rules
stephmilovic Dec 1, 2022
9bd8801
Merge branch 'main' into on_week_timeline_rules
stephmilovic Dec 1, 2022
0298d2d
fix cypress
stephmilovic Dec 1, 2022
2ff4713
fix cypress actually
stephmilovic Dec 2, 2022
bd8db38
Merge branch 'main' into on_week_timeline_rules
stephmilovic Dec 5, 2022
b6764ff
last fixes
stephmilovic Dec 5, 2022
e0aafad
fixes
stephmilovic Dec 5, 2022
0187e4b
fix tests
stephmilovic Dec 6, 2022
7503b74
Merge branch 'main' into on_week_timeline_rules
stephmilovic Dec 6, 2022
77e76d0
fix type
stephmilovic Dec 6, 2022
e67a962
pr fixes
stephmilovic Dec 6, 2022
a02eba6
fixed
stephmilovic Dec 7, 2022
2e476b1
rm commented code
stephmilovic Dec 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ import {
goToActionsStepTab,
goToScheduleStepTab,
importSavedQuery,
removeAlertsIndex,
waitForAlertsToPopulate,
waitForTheRuleToBeExecuted,
} from '../../tasks/create_new_rule';
Expand Down Expand Up @@ -133,6 +134,7 @@ describe('Custom query rules', () => {

cy.log('Filling define section');
importSavedQuery(this.timelineId);
removeAlertsIndex();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behavior of import rule from saved timeline changed with this PR to match the index pattern of the timeline. Because the saved timeline used in tests has dataViewId: null and indexNames: [], we know that this is a pre-sourcerer timeline when we only used our default security data view, so we use that data view for this timeline.

Depending on if any alerts have been generated yet, this data view includes auditbeat-* and .alerts-security.alerts-default. We now need to check if alerts index exists and remove it if it does, or it will break tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation here. I worry that the information you shared here can be forgotten in the future so it won't be so clear why alerts index is cleared here. Is it possible to remove alerts index inside importSavedQuery()? If not I'd leave a short comment in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it looks like this is the only place its used, will do

continueWithNextSection();

cy.log('Filling about section');
Expand Down Expand Up @@ -215,10 +217,7 @@ describe('Custom query rules', () => {
cy.get(INVESTIGATION_NOTES_TOGGLE).click({ force: true });
cy.get(ABOUT_INVESTIGATION_NOTES).should('have.text', INVESTIGATION_NOTES_MARKDOWN);
cy.get(DEFINITION_DETAILS).within(() => {
getDetails(INDEX_PATTERNS_DETAILS).should(
'have.text',
ruleFields.defaultIndexPatterns.join('')
);
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', 'auditbeat-*');
getDetails(CUSTOM_QUERY_DETAILS).should('have.text', ruleFields.ruleQuery);
getDetails(RULE_TYPE_DETAILS).should('have.text', 'Query');
getDetails(TIMELINE_TEMPLATE_DETAILS).should('have.text', 'None');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import { formatMitreAttackDescription } from '../../helpers/rules';
import type { Mitre } from '../../objects/rule';
import { getNewTermsRule, getIndexPatterns } from '../../objects/rule';
import { getNewTermsRule } from '../../objects/rule';

import { ALERT_DATA_GRID } from '../../screens/alerts';
import {
Expand Down Expand Up @@ -128,7 +128,7 @@ describe('New Terms rules', () => {
cy.get(INVESTIGATION_NOTES_TOGGLE).click({ force: true });
cy.get(ABOUT_INVESTIGATION_NOTES).should('have.text', INVESTIGATION_NOTES_MARKDOWN);
cy.get(DEFINITION_DETAILS).within(() => {
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', getIndexPatterns().join(''));
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', 'auditbeat-*');
getDetails(CUSTOM_QUERY_DETAILS).should('have.text', this.rule.customQuery);
getDetails(RULE_TYPE_DETAILS).should('have.text', 'New Terms');
getDetails(TIMELINE_TEMPLATE_DETAILS).should('have.text', 'None');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import { formatMitreAttackDescription } from '../../helpers/rules';
import type { Mitre, OverrideRule } from '../../objects/rule';
import { getIndexPatterns, getNewOverrideRule, getSeveritiesOverride } from '../../objects/rule';
import { getNewOverrideRule, getSeveritiesOverride } from '../../objects/rule';
import type { CompleteTimeline } from '../../objects/timeline';

import { NUMBER_OF_ALERTS, ALERT_GRID_CELL } from '../../screens/alerts';
Expand Down Expand Up @@ -146,7 +146,7 @@ describe('Detection rules, override', () => {
cy.get(INVESTIGATION_NOTES_TOGGLE).click({ force: true });
cy.get(ABOUT_INVESTIGATION_NOTES).should('have.text', INVESTIGATION_NOTES_MARKDOWN);
cy.get(DEFINITION_DETAILS).within(() => {
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', getIndexPatterns().join(''));
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', 'auditbeat-*');
getDetails(CUSTOM_QUERY_DETAILS).should('have.text', this.rule.customQuery);
getDetails(RULE_TYPE_DETAILS).should('have.text', 'Query');
getDetails(TIMELINE_TEMPLATE_DETAILS).should('have.text', 'None');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import { formatMitreAttackDescription } from '../../helpers/rules';
import type { Mitre } from '../../objects/rule';
import { getIndexPatterns, getNewThresholdRule } from '../../objects/rule';
import { getNewThresholdRule } from '../../objects/rule';

import { ALERT_GRID_CELL, NUMBER_OF_ALERTS } from '../../screens/alerts';

Expand Down Expand Up @@ -124,7 +124,7 @@ describe('Detection rules, threshold', () => {
cy.get(INVESTIGATION_NOTES_TOGGLE).click({ force: true });
cy.get(ABOUT_INVESTIGATION_NOTES).should('have.text', INVESTIGATION_NOTES_MARKDOWN);
cy.get(DEFINITION_DETAILS).within(() => {
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', getIndexPatterns().join(''));
getDetails(INDEX_PATTERNS_DETAILS).should('have.text', 'auditbeat-*');
getDetails(CUSTOM_QUERY_DETAILS).should('have.text', rule.customQuery);
getDetails(RULE_TYPE_DETAILS).should('have.text', 'Threshold');
getDetails(TIMELINE_TEMPLATE_DETAILS).should('have.text', 'None');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -228,3 +228,8 @@ export const savedQueryByName = (savedQueryName: string) =>

export const APPLY_SELECTED_SAVED_QUERY_BUTTON =
'[data-test-subj="saved-query-management-apply-changes-button"]';

export const RULE_INDICES =
'[data-test-subj="detectionEngineStepDefineRuleIndices"] [data-test-subj="comboBoxInput"]';

export const ALERTS_INDEX_BUTTON = 'span[title=".alerts-security.alerts-default"] button';
33 changes: 16 additions & 17 deletions x-pack/plugins/security_solution/cypress/tasks/create_new_rule.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ import {
ACTIONS_THROTTLE_INPUT,
CONTINUE_BUTTON,
CREATE_WITHOUT_ENABLING_BTN,
RULE_INDICES,
ALERTS_INDEX_BUTTON,
} from '../screens/create_new_rule';
import {
INDEX_SELECTOR,
Expand Down Expand Up @@ -344,13 +346,27 @@ const fillCustomQuery = (rule: CustomRule | OverrideRule) => {
cy.get(IMPORT_QUERY_FROM_SAVED_TIMELINE_LINK).click();
cy.get(TIMELINE(rule.timeline.id)).click();
cy.get(CUSTOM_QUERY_INPUT).should('have.value', rule.customQuery);
if (rule.dataSource.type === 'indexPatterns') {
removeAlertsIndex();
}
} else {
cy.get(CUSTOM_QUERY_INPUT)
.first()
.type(rule.customQuery || '');
}
};

// called after import rule from saved timeline
// if alerts index is created, it is included in the timeline
// to be consistent in multiple test runs, remove it if it's there
export const removeAlertsIndex = () => {
cy.get(RULE_INDICES).then(($body) => {
if ($body.find(ALERTS_INDEX_BUTTON).length > 0) {
cy.get(ALERTS_INDEX_BUTTON).click();
}
});
};

export const continueWithNextSection = () => {
cy.get(CONTINUE_BUTTON).should('exist').click();
};
Expand Down Expand Up @@ -690,20 +706,3 @@ export const checkLoadQueryDynamically = () => {
export const uncheckLoadQueryDynamically = () => {
cy.get(LOAD_QUERY_DYNAMICALLY_CHECKBOX).click({ force: true }).should('not.be.checked');
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused

export const defineSection = { importSavedQuery };
export const aboutSection = {
fillRuleName,
fillDescription,
fillSeverity,
fillRiskScore,
fillRuleTags,
expandAdvancedSettings,
fillReferenceUrls,
fillFalsePositiveExamples,
fillThreat,
fillThreatTechnique,
fillThreatSubtechnique,
fillNote,
};
export const scheduleSection = { fillFrom };
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,6 @@ export const RULE_PREVIEW_TITLE = i18n.translate(
}
);

export const RULE_PREVIEW_DESCRIPTION = i18n.translate(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused

'xpack.securitySolution.detectionEngine.createRule.rulePreviewDescription',
{
defaultMessage:
'Rule preview reflects the current configuration of your rule settings and exceptions, click refresh icon to see the updated preview.',
}
);

export const CANCEL_BUTTON_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.cancelButtonLabel',
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
*/

import React from 'react';
import { mount } from 'enzyme';

import { QueryBarDefineRule } from '.';
import {
Expand All @@ -16,7 +15,11 @@ import {
} from '../../../../common/mock';
import { useGetAllTimeline, getAllTimeline } from '../../../../timelines/containers/all';
import { mockHistory, Router } from '../../../../common/mock/router';
import { render, act, fireEvent } from '@testing-library/react';
import { resolveTimeline } from '../../../../timelines/containers/api';
import { mockTimeline } from '../../../../../server/lib/timeline/__mocks__/create_timelines';

jest.mock('../../../../timelines/containers/api');
jest.mock('../../../../common/lib/kibana', () => {
const actual = jest.requireActual('../../../../common/lib/kibana');
return {
Expand Down Expand Up @@ -48,68 +51,95 @@ jest.mock('../../../../timelines/containers/all', () => {

describe('QueryBarDefineRule', () => {
beforeEach(() => {
jest.clearAllMocks();
(useGetAllTimeline as unknown as jest.Mock).mockReturnValue({
fetchAllTimeline: jest.fn(),
timelines: getAllTimeline('', mockOpenTimelineQueryResults.timeline ?? []),
loading: false,
totalCount: mockOpenTimelineQueryResults.totalCount,
refetch: jest.fn(),
});
(resolveTimeline as jest.Mock).mockResolvedValue({
data: {
timeline: { mockTimeline },
},
});
});

it('renders correctly', () => {
const Component = () => {
const field = useFormFieldMock();
const field = useFormFieldMock();

return (
<QueryBarDefineRule
browserFields={{}}
isLoading={false}
indexPattern={{ fields: [], title: 'title' }}
onCloseTimelineSearch={jest.fn()}
openTimelineSearch={true}
dataTestSubj="query-bar-define-rule"
idAria="idAria"
field={field}
/>
);
};
const wrapper = mount(
const { getByTestId } = render(
<TestProviders>
<Router history={mockHistory}>
<Component />
<QueryBarDefineRule
browserFields={{}}
isLoading={false}
indexPattern={{ fields: [], title: 'title' }}
onCloseTimelineSearch={jest.fn()}
openTimelineSearch={true}
dataTestSubj="query-bar-define-rule"
idAria="idAria"
field={field}
/>
</Router>
</TestProviders>
);
expect(wrapper.find('[data-test-subj="query-bar-define-rule"]').exists()).toBeTruthy();
expect(getByTestId('query-bar-define-rule')).toBeInTheDocument();
});

it('renders import query from saved timeline modal actions hidden correctly', () => {
const Component = () => {
it('renders import query from saved timeline modal actions hidden correctly', async () => {
await act(async () => {
const field = useFormFieldMock();

return (
<QueryBarDefineRule
browserFields={{}}
isLoading={false}
indexPattern={{ fields: [], title: 'title' }}
onCloseTimelineSearch={jest.fn()}
openTimelineSearch={true}
dataTestSubj="query-bar-define-rule"
idAria="idAria"
field={field}
/>
const { queryByTestId } = render(
<TestProviders>
<Router history={mockHistory}>
<QueryBarDefineRule
browserFields={{}}
isLoading={false}
indexPattern={{ fields: [], title: 'title' }}
onCloseTimelineSearch={jest.fn()}
openTimelineSearch={true}
dataTestSubj="query-bar-define-rule"
idAria="idAria"
field={field}
/>
</Router>
</TestProviders>
);
};
const wrapper = mount(

expect(queryByTestId('open-duplicate')).not.toBeInTheDocument();
expect(queryByTestId('create-from-template')).not.toBeInTheDocument();
});
});

it('calls onOpenTimeline correctly', async () => {
const field = useFormFieldMock();
const onOpenTimeline = jest.fn();

const { getByTestId } = render(
<TestProviders>
<Router history={mockHistory}>
<Component />
<QueryBarDefineRule
browserFields={{}}
isLoading={false}
indexPattern={{ fields: [], title: 'title' }}
onCloseTimelineSearch={jest.fn()}
openTimelineSearch={true}
dataTestSubj="query-bar-define-rule"
idAria="idAria"
field={field}
onOpenTimeline={onOpenTimeline}
/>
</Router>
</TestProviders>
);
getByTestId('open-timeline-modal').click();

expect(wrapper.find('[data-test-subj="open-duplicate"]').exists()).toBeFalsy();
expect(wrapper.find('[data-test-subj="create-from-template"]').exists()).toBeFalsy();
await act(async () => {
fireEvent.click(getByTestId('title-10849df0-7b44-11e9-a608-ab3d811609'));
});
expect(onOpenTimeline).toHaveBeenCalled();
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@ import type { BrowserFields } from '../../../../common/containers/source';
import { OpenTimelineModal } from '../../../../timelines/components/open_timeline/open_timeline_modal';
import type { ActionTimelineToShow } from '../../../../timelines/components/open_timeline/types';
import { QueryBar } from '../../../../common/components/query_bar';
import { buildGlobalQuery } from '../../../../timelines/components/timeline/helpers';
import { getDataProviderFilter } from '../../../../timelines/components/timeline/query_bar';
import { convertKueryToElasticSearchQuery } from '../../../../common/lib/kuery';
import { useKibana } from '../../../../common/lib/kibana';
import type { TimelineModel } from '../../../../timelines/store/timeline/model';
import { useSavedQueryServices } from '../../../../common/utils/saved_query_services';
Expand Down Expand Up @@ -54,6 +51,7 @@ export interface QueryBarDefineRuleProps {
*/
onSavedQueryError?: () => void;
defaultSavedQuery?: SavedQuery | undefined;
onOpenTimeline?: (timeline: TimelineModel) => void;
}

const actionTimelineToHide: ActionTimelineToShow[] = ['duplicate', 'createFrom'];
Expand Down Expand Up @@ -88,6 +86,7 @@ export const QueryBarDefineRule = ({
onValidityChange,
isDisabled,
resetToSavedQuery,
onOpenTimeline,
onSavedQueryError,
}: QueryBarDefineRuleProps) => {
const { value: fieldValue, setValue: setFieldValue } = field as FieldHook<FieldValueQueryBar>;
Expand Down Expand Up @@ -234,31 +233,12 @@ export const QueryBarDefineRule = ({
onCloseTimelineSearch();
}, [onCloseTimelineSearch]);

const onOpenTimeline = useCallback(
const onOpenTimelineCb = useCallback(
(timeline: TimelineModel) => {
setLoadingTimeline(false);
const newQuery = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logic gets moved to new hook, use_rule_from_timeline.tsx

query: timeline.kqlQuery.filterQuery?.kuery?.expression ?? '',
language: timeline.kqlQuery.filterQuery?.kuery?.kind ?? 'kuery',
};
const dataProvidersDsl =
timeline.dataProviders != null && timeline.dataProviders.length > 0
? convertKueryToElasticSearchQuery(
buildGlobalQuery(timeline.dataProviders, browserFields),
indexPattern
)
: '';
const newFilters = timeline.filters ?? [];
setFieldValue({
filters:
dataProvidersDsl !== ''
? [...newFilters, getDataProviderFilter(dataProvidersDsl)]
: newFilters,
query: newQuery,
saved_id: null,
});
if (onOpenTimeline != null) onOpenTimeline(timeline);
stephmilovic marked this conversation as resolved.
Show resolved Hide resolved
},
[browserFields, indexPattern, setFieldValue]
[onOpenTimeline]
);

const onMutation = () => {
Expand Down Expand Up @@ -324,7 +304,7 @@ export const QueryBarDefineRule = ({
hideActions={actionTimelineToHide}
modalTitle={i18n.IMPORT_TIMELINE_MODAL}
onClose={onCloseTimelineModal}
onOpen={onOpenTimeline}
onOpen={onOpenTimelineCb}
/>
) : null}
</>
Expand Down