[ML] Quickly create ML jobs from lens visualizations #136421

jgowdyelastic · 2022-07-14T16:34:21Z

Anomaly detection jobs can now be created instantly from the flyout within the dashboard app.
The original option to create via the wizard is also still available as a link to the ML app.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
[ x] This was checked for breaking API changes and was labeled appropriately

…com:jgowdyelastic/kibana into quickly-create-jobs-from-lens-visualization

elasticmachine · 2022-08-10T15:54:58Z

Pinging @elastic/ml-ui (:ml)

peteharverson

Tested latest changes and LGTM

jgowdyelastic · 2022-08-25T09:01:41Z

@elastic/kibana-design you have been pinged because I have deleted a scss file.

darnautov

Great job with refactoring, overall LGTM! Left some minor suggestions

darnautov · 2022-08-25T08:42:59Z

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/quick_create_job.ts

+    startJob: boolean,
+    runInRealTime: boolean,
+    layerIndex: number,
+    mlApiServices: MlApiServices


nit: I see it's only been used by createAndSaveJob method, but maybe it's worth passing to a constructor instead?

in the last refactor I moved mlApiServices to the context after changing this, so possibly yes.
Previously it wasn't available in all of the locations QuickJobCreator was instantiated.

Changed in 336db0c

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/quick_create_job.ts

darnautov · 2022-08-25T08:53:51Z

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/quick_create_job.ts

+      // add job config and start and end dates to the
+      // job cloning stash, so they can be used
+      // by the new job wizards
+      stashJobForCloning(


Checked the content of this function an noticed it utilizes mlJobService which is statically exported. As an idea for further enhancements: I reckon we should provide this service from the plugin contract setup, so the other plugins can consume it as well.

mlJobService is very old code and shouldn't be shared.
Most of it has been replaced by other, better, services. I believe it is only the results pages which still use it.

The most useful part is the job stash used for cloning. So ideally we should get rid of mlJobService and move the cloning functionality to a separate service.

darnautov · 2022-08-25T08:55:12Z

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/quick_create_job.ts

+      }
+    } catch (error) {
+      // eslint-disable-next-line no-console
+      console.error(error);


❓ do we want to show an error toast here?

I think it is ok to fail silently here.
includeTimeRange is set to false which means the wizard will not auto set the time. The user can then choose one.

darnautov · 2022-08-25T09:34:20Z

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/utils.ts

+    },
+  });
+
+  window.open(url, '_blank');


Please use navigateToUrl from CoreStart['application']

This link needs to open in a new tab, from what I can tell navigateToUrl does not allow this as an option.

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/utils.ts

x-pack/plugins/ml/public/application/jobs/new_job/job_from_lens/visualization_extractor.ts

darnautov

Code LGTM

mdefazio

Looks good!
One bit of UI feedback would be to change the background of the panel in the flyout to white (where you define the job settings). This will help contrast it against the gray background of the flyout.
Was curious if it also made sense to add in the panel title for more context? Instead of only the layer type?

jgowdyelastic · 2022-08-25T15:35:56Z

Thanks @mdefazio , I've added the visualisation title to the description at the top of the flyout. 65289fd

Regarding the background colour, I copied the style used in Lens:

Changing the colour in my panels does make it standout from the background, but IMO they look a bit too white and strange.

vs the original

mdefazio · 2022-08-25T16:48:02Z

Thanks @jgowdyelastic , is there a scenario where you're setting up 2+ jobs?

The one difference between Lens is that the input or area of focus is still on white (@timestamp or Median of CPUtilization is in a white panel over the gray). Granted, because you're including the input field directly in place of the inner panel, it's a bit up in the air as to how to present this. But simply for contrast on the input, the white might be worth it.

jgowdyelastic · 2022-08-25T19:25:42Z

@mdefazio

is there a scenario where you're setting up 2+ jobs?

If you have multiple compatible layers in a visualization it would be possible to create multiple jobs from a single flyout. The layer components are independent from each other.
It's worth noting, in case there is come confusion here, that the layers themselves are not named. The best I can do is use the layer types as the title (e.g. stacked bar chart)
This was raised before in the previous PR #129762 (comment) (second half of this comment). It would be better to have a more descriptive title for the layer, but it's not easy, or even possible.

mdefazio · 2022-08-25T21:07:22Z

Ok...now it's coming back to me! Thanks for the reminder of the history / context!

…com:jgowdyelastic/kibana into quickly-create-jobs-from-lens-visualization

kibana-ci · 2022-08-26T07:49:38Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: a2330ff

Failed CI Steps

FTR Configs #27

Test Failures

[job] [logs] FTR Configs #27 / a11y tests using flights sample data "after all" hook in "using flights sample data"
[job] [logs] FTR Configs #27 / a11y tests using flights sample data "before all" hook in "using flights sample data"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`ml`	1606	1605	-1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`ml`	3.3MB	3.3MB	+9.4KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`ml`	98	101	+3

Total ESLint disabled count

id	before	after	diff
`ml`	101	104	+3

History

💔 Build #67590 failed d9fd4b8
💔 Build #67522 failed 67064d7
💔 Build #67424 failed 65289fd
💚 Build #67334 succeeded 336db0c
💛 Build #67298 was flaky 18fa40e
💚 Build #67156 succeeded 563dc19

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic

* [ML] Quickly create ML jobs from lens visualizations * adding additional settings * adding model plot for single metric jobs * adding results link * form logic improvements * translations * translations * translations * changing view results link generation * handling opening already open jobs * fixing count detector * removing unused file * improving error displaying * updating tests * removing unused test code * deleting jobs after test * code clean up * fixing issue with layers sharing resources * removing fragment * moving code about * minor changes based on review * translations * updating results link for non-running job * renaming variables * refactoring job creation code into class * using context refactor * renaming things * catching isCompatible errors * fixing behaviour in canvas * removing comment * small refactor moving code * handling error * fixing typo * further refactoring based on review * adding vis title to description * white layer panels * translations

[ML] Quickly create ML jobs from lens visualizations

5f071b9

jgowdyelastic self-assigned this Jul 14, 2022

jgowdyelastic added 16 commits July 14, 2022 21:19

adding additional settings

2c5ec20

adding model plot for single metric jobs

ca6a451

Merge branch 'main' into quickly-create-jobs-from-lens-visualization

9e8c67a

adding results link

7612f53

form logic improvements

b0894be

Merge branch 'main' into quickly-create-jobs-from-lens-visualization

8e8371c

translations

039d509

Merge branch 'quickly-create-jobs-from-lens-visualization' of github.…

d2479bf

…com:jgowdyelastic/kibana into quickly-create-jobs-from-lens-visualization

translations

58b7e1d

translations

91b2cb5

changing view results link generation

ed096d1

handling opening already open jobs

a2556f3

fixing count detector

028c2f7

removing unused file

40b20a0

improving error displaying

374f014

updating tests

4b5c853

jgowdyelastic marked this pull request as ready for review August 10, 2022 15:00

jgowdyelastic requested a review from a team as a code owner August 10, 2022 15:00

jgowdyelastic changed the title ~~[ML] Quickly create ML jobs from lens visualizations~~ [ML] [WIP] Quickly create ML jobs from lens visualizations Aug 10, 2022

Merge branch 'main' into quickly-create-jobs-from-lens-visualization

52608c7

jgowdyelastic added release_note:enhancement :ml labels Aug 10, 2022

jgowdyelastic added Feature:Anomaly Detection ML anomaly detection v8.5.0 labels Aug 10, 2022

jgowdyelastic added 2 commits August 10, 2022 17:04

removing unused test code

2de87f9

deleting jobs after test

0d7fde5

jgowdyelastic marked this pull request as draft August 11, 2022 08:52

jgowdyelastic added 8 commits August 24, 2022 14:45

using context refactor

ea4a3f4

renaming things

273d85d

catching isCompatible errors

b073bf9

fixing behaviour in canvas

6bd7b1a

removing comment

54f7e62

small refactor moving code

3007004

handling error

563dc19

fixing typo

18fa40e

peteharverson approved these changes Aug 25, 2022

View reviewed changes

darnautov reviewed Aug 25, 2022

View reviewed changes

jgowdyelastic added 2 commits August 25, 2022 11:57

Merge branch 'main' into quickly-create-jobs-from-lens-visualization

c187371

further refactoring based on review

336db0c

darnautov approved these changes Aug 25, 2022

View reviewed changes

mdefazio approved these changes Aug 25, 2022

View reviewed changes

adding vis title to description

65289fd

white layer panels

67064d7

jgowdyelastic added 3 commits August 26, 2022 07:35

Merge branch 'main' into quickly-create-jobs-from-lens-visualization

d9fd4b8

translations

dd0423a

Merge branch 'quickly-create-jobs-from-lens-visualization' of github.…

a2330ff

…com:jgowdyelastic/kibana into quickly-create-jobs-from-lens-visualization

jgowdyelastic enabled auto-merge (squash) August 26, 2022 06:49

jgowdyelastic merged commit 4f65c09 into elastic:main Aug 26, 2022

kibanamachine added the backport:skip This commit does not require backporting label Aug 26, 2022

alvarezmelissa87 mentioned this pull request Dec 20, 2022

[ML] Quickly create ML jobs from map visualizations in dashboard #147882

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Quickly create ML jobs from lens visualizations #136421

[ML] Quickly create ML jobs from lens visualizations #136421

jgowdyelastic commented Jul 14, 2022 •

edited by peteharverson

elasticmachine commented Aug 10, 2022

peteharverson left a comment

jgowdyelastic commented Aug 25, 2022 •

edited

darnautov left a comment

darnautov Aug 25, 2022

jgowdyelastic Aug 25, 2022

jgowdyelastic Aug 25, 2022

darnautov Aug 25, 2022

jgowdyelastic Aug 25, 2022

darnautov Aug 25, 2022

jgowdyelastic Aug 25, 2022 •

edited

darnautov Aug 25, 2022

jgowdyelastic Aug 25, 2022

darnautov left a comment

mdefazio left a comment

jgowdyelastic commented Aug 25, 2022 •

edited

mdefazio commented Aug 25, 2022

jgowdyelastic commented Aug 25, 2022 •

edited

mdefazio commented Aug 25, 2022

kibana-ci commented Aug 26, 2022

ESLint disabled line counts

Total ESLint disabled count

[ML] Quickly create ML jobs from lens visualizations #136421

[ML] Quickly create ML jobs from lens visualizations #136421

Conversation

jgowdyelastic commented Jul 14, 2022 • edited by peteharverson

elasticmachine commented Aug 10, 2022

peteharverson left a comment

Choose a reason for hiding this comment

jgowdyelastic commented Aug 25, 2022 • edited

darnautov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgowdyelastic Aug 25, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darnautov left a comment

Choose a reason for hiding this comment

mdefazio left a comment

Choose a reason for hiding this comment

jgowdyelastic commented Aug 25, 2022 • edited

mdefazio commented Aug 25, 2022

jgowdyelastic commented Aug 25, 2022 • edited

mdefazio commented Aug 25, 2022

kibana-ci commented Aug 26, 2022

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Async chunks

ESLint disabled line counts

Total ESLint disabled count

History

jgowdyelastic commented Jul 14, 2022 •

edited by peteharverson

jgowdyelastic commented Aug 25, 2022 •

edited

jgowdyelastic Aug 25, 2022 •

edited

jgowdyelastic commented Aug 25, 2022 •

edited

jgowdyelastic commented Aug 25, 2022 •

edited