[ML] Add functional tests for Data Drift view #167911

qn895 · 2023-10-03T16:17:21Z

Summary

This PR adds functional test for Data drift view. It covers the following scenarios:

From index or saved search selection page, open data drift for KQL saved search
- Check doc count reflects the query & filters
- Check table exists
From data view creation view, open data drift view for time series dataset without creating new data view
From data view creation view, save new data view, and open data drift view for timeseries dataset
From data view creation view, save new data view, and open data drift view for non-timeseries dataset
From Trained models list' available actions, open data drift's data view creation view, open data drift view for timeseries

Flaky test suite runner... 49/50 passed ✅ - failed test is an unrelated flaky test

PR also fixes:

When ks test returns NaN value, this happens when the distributions are non overlapping. This means there is a drift and the p-value would be astronomically small. This PR enforces so that it shows Drift detected as 'Yes' and value < 0.000001.

Checklist

Unit or functional tests were updated or added to match the most common scenarios

…tests

elasticmachine · 2023-10-10T04:40:34Z

Pinging @elastic/ml-ui (:ml)

peteharverson

LGTM, although looks like one of the flaky test runner failed.

x-pack/test/functional/apps/ml/short_tests/model_management/model_list.ts

qn895 · 2023-10-10T16:18:55Z

LGTM, although looks like one of the flaky test runner failed.

The failed test is actually not related to data drift 🤔 Looks like a flaky test, need some more investigation.

…tests

darnautov · 2023-10-11T06:05:29Z

x-pack/plugins/data_visualizer/public/application/data_drift/data_drift_overview_table.tsx

@@ -149,7 +152,9 @@ export const DataDriftOverviewTable = ({
      'data-test-subj': 'mlDataDriftOverviewTableDriftDetected',
      sortable: true,
      textOnly: true,
-      render: (driftDetected: boolean) => {
+      render: (driftDetected: boolean, item) => {
+        // @ts-expect-error currently ES two_sided does return string NaN, will be fixed


is there an issue we can mention in the comment?

Valeriy is investigating this and will create an issue on the Elasticsearch side. But for now, we don't have an issue.

darnautov · 2023-10-11T08:27:48Z

x-pack/test/functional/apps/ml/data_visualizer/data_drift.ts

-      );
-      await ml.dataDrift.assertNoWindowParametersEmptyPromptExists();
+    await ml.testExecution.logTestStep(
+      `${testData.suiteTitle} displays elements in the page correctly`


Suggested change

`${testData.suiteTitle} displays elements in the page correctly`

`${testData.suiteTitle} displays elements on the page correctly`

Updated here 0ef6b56 (#167911)

darnautov · 2023-10-11T08:49:06Z

x-pack/test/functional/services/ml/data_drift.ts

+        const btn = await testSubjects.find('analyzeDataDriftButton');
+        const isDisabled = await btn.getAttribute('disabled');
+        expect(isDisabled).to.equal(disabled ? 'true' : null);


Suggested change

const btn = await testSubjects.find('analyzeDataDriftButton');

const isDisabled = await btn.getAttribute('disabled');

expect(isDisabled).to.equal(disabled ? 'true' : null);

const isDisabled = !(awaot testSubjects. isEnabled('analyzeDataDriftButton));

expect(isDisabled).to.equal(disabled);

Updated here 0ef6b56 (#167911)

darnautov · 2023-10-11T08:50:00Z

x-pack/test/functional/services/ml/data_drift.ts

+        const btn = await testSubjects.find('analyzeDataDriftWithoutSavingButton');
+        const isDisabled = await btn.getAttribute('disabled');
+        expect(isDisabled).to.equal(disabled ? 'true' : null);


Suggested change

const btn = await testSubjects.find('analyzeDataDriftWithoutSavingButton');

const isDisabled = await btn.getAttribute('disabled');

expect(isDisabled).to.equal(disabled ? 'true' : null);

const isDisabled = !(await testSubjects.isEnabled('analyzeDataDriftWithoutSavingButton'));

expect(isDisabled).to.equal(disabled);

Updated here 0ef6b56 (#167911)

darnautov · 2023-10-11T08:54:39Z

x-pack/test/functional/services/ml/trained_models_table.ts

+      });
+    }
+
+    public async clickAnalyzeDataDriftActionButton(modelId: string) {


You can utiilze the invokeAction method from this service instead

invokeAction doesn't seem to account for when button is directly on the row (e.g. when there are <= 2 actions), or is hidden by the context menu (> 2 actions available) 🤔

darnautov · 2023-10-11T08:59:56Z

x-pack/test/functional/services/ml/trained_models_table.ts

+      );
+    }
+
+    public async assertAnalyzeDataDriftActionButtonEnabled(


would be great to have a reusable assertRowActionEnabled method in generic table service

I agree that we should have a reusable assertRowActionEnabled, but the current generic table service doesn't have concept for a context menu for the actions. That would require a lot of refactoring for the current trained models table & the generic service. I think that's out of scope of this PR.

…tests

qn895 · 2023-10-12T14:28:33Z

@elasticmachine merge upstream

peteharverson

Tested with fix for item 1 in #168090, and LGTM

qn895 · 2023-10-12T18:39:59Z

@elasticmachine merge upstream

kibana-ci · 2023-10-12T19:46:21Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 5a4a6d4

Failed CI Steps

Serverless Osquery Cypress Tests #6

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`dataVisualizer`	613.6KB	613.8KB	+205.0B
`ml`	3.5MB	3.5MB	+301.0B
total			+506.0B

History

💔 Build #167349 failed f180ede
💔 Build #167060 failed 0ef6b56
💔 Build #166762 failed 6f0bde3
💚 Build #166702 succeeded f2237b6
💛 Build #166473 was flaky f7b86cb

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @qn895

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit f28349f)

kibanamachine · 2023-10-12T22:59:57Z

💚 All backports created successfully

Status	Branch	Result
✅	8.11

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.11`: - [[ML] Add functional tests for Data Drift view (#167911)](#167911)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Quynh Nguyen (Quinn) <43350163+qn895@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

qn895 added the :ml label Oct 3, 2023

qn895 self-assigned this Oct 3, 2023

qn895 added test_ui_functional v8.11.0 v8.11.1 labels Oct 3, 2023

Add tests

d441274

qn895 force-pushed the ml-data-drift-func-tests branch from 2e1e962 to d441274 Compare October 3, 2023 17:00

Fix test data type

90a6a7a

peteharverson mentioned this pull request Oct 4, 2023

[ML] Increase Test Coverage 8.11.0 #164562

Closed

10 tasks

qn895 added 6 commits October 4, 2023 10:07

Merge remote-tracking branch 'upstream/main' into ml-data-drift-func-…

6644959

…tests

Assert time field

7e99182

Merge remote-tracking branch 'upstream/main' into ml-data-drift-func-…

687a8ea

…tests

Assert tests for data drift from trained model page

92494a2

Assert tests for data drift from trained model page

d7b5906

Revert input to append wildcard

9460e16

qn895 force-pushed the ml-data-drift-func-tests branch 3 times, most recently from 887457f to c6642fd Compare October 9, 2023 19:39

qn895 requested review from alvarezmelissa87 and removed request for alvarezmelissa87 October 9, 2023 20:45

Stabilize tests

f7b86cb

qn895 force-pushed the ml-data-drift-func-tests branch from c6642fd to f7b86cb Compare October 9, 2023 21:39

qn895 marked this pull request as ready for review October 10, 2023 04:40

qn895 requested a review from a team as a code owner October 10, 2023 04:40

qn895 requested review from alvarezmelissa87 and darnautov October 10, 2023 04:40

peteharverson approved these changes Oct 10, 2023

View reviewed changes

x-pack/test/functional/apps/ml/short_tests/model_management/model_list.ts Outdated Show resolved Hide resolved

Fix //

f2237b6

qn895 added the release_note:skip Skip the PR/issue when compiling release notes label Oct 10, 2023

qn895 added 2 commits October 10, 2023 12:33

Hide NaN

5a9f89a

Merge remote-tracking branch 'upstream/main' into ml-data-drift-func-…

6f0bde3

…tests

darnautov reviewed Oct 11, 2023

View reviewed changes

qn895 added 3 commits October 11, 2023 09:38

Merge remote-tracking branch 'upstream/main' into ml-data-drift-func-…

e7d668b

…tests

Update NaN

820da05

Fix disabled state

0ef6b56

qn895 requested a review from darnautov October 11, 2023 21:26

Merge branch 'main' into ml-data-drift-func-tests

f180ede

peteharverson approved these changes Oct 12, 2023

View reviewed changes

Merge branch 'main' into ml-data-drift-func-tests

5a4a6d4

qn895 merged commit f28349f into elastic:main Oct 12, 2023
34 checks passed

kibanamachine added the v8.12.0 label Oct 12, 2023

kibanamachine mentioned this pull request Oct 12, 2023

[8.11] [ML] Add functional tests for Data Drift view (#167911) #168756

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 12, 2023

[ML] Add functional tests for Data Drift view (elastic#167911)

92d646e

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit f28349f)

dej611 pushed a commit to dej611/kibana that referenced this pull request Oct 17, 2023

[ML] Add functional tests for Data Drift view (elastic#167911)

2f9a728

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

qn895 mentioned this pull request Oct 19, 2023

[ML] Data Drift View bugs #168090

Closed

benakansara pushed a commit to benakansara/kibana that referenced this pull request Oct 22, 2023

[ML] Add functional tests for Data Drift view (elastic#167911)

9b639f5

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

benakansara pushed a commit to benakansara/kibana that referenced this pull request Oct 22, 2023

[ML] Add functional tests for Data Drift view (elastic#167911)

827842e

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Add functional tests for Data Drift view #167911

[ML] Add functional tests for Data Drift view #167911

qn895 commented Oct 3, 2023 •

edited by kibanamachine

elasticmachine commented Oct 10, 2023

peteharverson left a comment

qn895 commented Oct 10, 2023

darnautov Oct 11, 2023

qn895 Oct 11, 2023

darnautov Oct 11, 2023

qn895 Oct 11, 2023

darnautov Oct 11, 2023

qn895 Oct 11, 2023

darnautov Oct 11, 2023

qn895 Oct 11, 2023

darnautov Oct 11, 2023

qn895 Oct 11, 2023

darnautov Oct 11, 2023

qn895 Oct 11, 2023

qn895 commented Oct 12, 2023

peteharverson left a comment

qn895 commented Oct 12, 2023

kibana-ci commented Oct 12, 2023

kibanamachine commented Oct 12, 2023

	`${testData.suiteTitle} displays elements in the page correctly`
	`${testData.suiteTitle} displays elements on the page correctly`

[ML] Add functional tests for Data Drift view #167911

[ML] Add functional tests for Data Drift view #167911

Conversation

qn895 commented Oct 3, 2023 • edited by kibanamachine

Summary

Checklist

elasticmachine commented Oct 10, 2023

peteharverson left a comment

Choose a reason for hiding this comment

qn895 commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qn895 commented Oct 12, 2023

peteharverson left a comment

Choose a reason for hiding this comment

qn895 commented Oct 12, 2023

kibana-ci commented Oct 12, 2023

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

History

kibanamachine commented Oct 12, 2023

💚 All backports created successfully

Questions ?

qn895 commented Oct 3, 2023 •

edited by kibanamachine