[Dashboard] [Controls] Add unmapped runtime field support to options list #144947

Heenawter · 2022-11-09T21:05:49Z

Summary

This PR adds options list support for runtime fields that are added to data views - note that runtime fields that are added as part of an index (i.e. they are part of the mapping) already worked as expected.

RuntimeFields.mov

How to Test

Create some data view runtime fields of various types as described in the documentation. For example, in the sample Logs data view, you could create the following:

Keyword:

emit(doc['referer'].value.splitOnToken('/')[2].replace(' ', ''))

Keyword with partial data (to test the exists options list functionality):

if(doc['geo.dest'].value == 'US') {
    emit('Party in the USA')
}

Boolean:

emit(doc['geo.src'].value == doc['geo.dest'].value)

IP:
```
emit(doc['clientip'].value)
```

Composite:

if(doc['geo.dest'].value == doc['geo.src'].value) {
    emit('english', 'equals');
    emit('spanish', 'es igual');
    emit('french', 'équivaut à');
    return;
}
  
emit('english', 'does not equal');
emit('spanish', 'no es igual');
emit('french', 'n\'est pas égal');

Create options list controls on these new fields
Ensure that they work as expected, including chaining and validation.

Flaky Test Runner

Checklist

Unit or functional tests were updated or added to match the most common scenarios
This was checked for cross-browser compatibility

For maintainers

This was checked for breaking API changes and was labeled appropriately

Heenawter · 2022-11-14T19:21:00Z

src/plugins/controls/public/control_group/editor/data_control_editor_tools.ts

+export const getDataControlFieldRegistry = memoize(
+  async (dataView: DataView) => {
+    return await loadFieldRegistryFromDataViewId(dataView);
+  },
+  (dataView: DataView) => [dataView.id, JSON.stringify(dataView.fields.getAll())].join('|')
+);


With the old "fake" cache, because it relied solely on data view IDs, adding a runtime field required a hard refresh in order to get this new field to show up in the field list for creating/editing controls. By using Lodash's memoize instead, I can force the cache to update when the fields change, so a hard refresh is no longer required when runtime fields are added.

Heenawter · 2022-11-14T19:24:10Z

test/functional/apps/dashboard_elements/controls/options_list.ts

+        });
+      });
+
+      describe('test exists query', async () => {


Since the runtime field tests also require adding data (but in a slightly different way), I decided to encapsulate the relevant data addition/removal in to the respective runtime field and exists query test suites rather than doing it all as part of the main test suite setup.

elasticmachine · 2022-11-14T19:25:32Z

Pinging @elastic/kibana-presentation (Team:Presentation)

nreese · 2022-11-14T21:41:39Z

src/plugins/controls/public/control_group/editor/data_control_editor_tools.ts

@@ -36,20 +44,13 @@ const doubleLinkFields = (dataView: DataView) => {
  return fieldRegistry;
 };

-export const loadFieldRegistryFromDataViewId = async (
-  dataViewId: string
+const loadFieldRegistryFromDataViewId = async (


nit - How about renaming loadFieldRegistryFromDataViewId -> loadFieldRegistryFromDataView?

Oh yup, good catch! I changed it to take the data view instead of the ID so that I could get memoization to work, and I forgot to update the name to reflect this change 👍

Fixed in b1fa3e1

nreese

LGTM
code review, tested in chrome

kibana-ci · 2022-11-14T22:55:18Z

💚 Build Succeeded

Buildkite Build
Commit: b1fa3e1

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`controls`	473.5KB	473.9KB	+344.0B

Unknown metric groups

ESLint disabled in files

id	before	after	diff
`osquery`	1	2	+1

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`fleet`	59	65	+6
`osquery`	108	113	+5
`securitySolution`	441	447	+6
total			+19

Total ESLint disabled count

id	before	after	diff
`enterpriseSearch`	20	22	+2
`fleet`	67	73	+6
`osquery`	109	115	+6
`securitySolution`	518	524	+6
total			+20

History

💚 Build #87704 succeeded 7c5f88c
💚 Build #87145 succeeded 57665cd
💔 Build #87083 failed cb917bc
💚 Build #86864 succeeded 2dc599c
💚 Build #86839 succeeded 9fed8e7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @Heenawter

* main: (65 commits) Migrate server-side `Root` and `Server` to packages (elastic#144990) [Discover] Handle no data views state for `esQuery` alert (elastic#145052) [ML] Allow updates for number of allocations and priority for trained model deployments (elastic#144704) [api-docs] 2022-11-15 Daily api_docs build (elastic#145203) [Security solution] remove guided onboarding feature flag (elastic#144247) [DOCS] Automate final case APIs (elastic#145007) [Enterprise Search] Name and description flyout for connectors (elastic#143827) [Guided onboarding] Update header button logic (elastic#144634) [Lens] Multi metric partition charts (elastic#143966) [Dashboard] [Controls] Add unmapped runtime field support to options list (elastic#144947) [Security Solution] Add Task Metric Collection to New Tasks (elastic#145181) [TriggersActionsUi] disable jest config in CI (elastic#145186) [TableListView] Enhance tag filtering (elastic#142108) [Cloud Posture] Compliance by CIS section table (elastic#145114) [8.6][Session View] Fix hidden alert flyout in session view (elastic#145141) [customIntegrations] async load all components (elastic#145166) Fix time for logs smoke tests in integration test (elastic#145130) [RAM] Update rule status (elastic#140882) Update babel (main) (elastic#145060) [Actionable Observability] Add context.alertDetailsUrl variable to action connector template for APM rule types (elastic#144791) ...

Heenawter self-assigned this Nov 9, 2022

Heenawter force-pushed the add-data-view-runtime-field-support_2022-11-09 branch 5 times, most recently from b9e4113 to 80f31fe Compare November 14, 2022 17:51

Heenawter added 4 commits November 14, 2022 12:20

Add data view run time field support

71ea7f6

Add support for composite runtime fields

3dd4dbc

Add validation support for runtime fields

586d594

Memoize field registry to avoid refresh bug

9e89d23

Heenawter force-pushed the add-data-view-runtime-field-support_2022-11-09 branch from 80f31fe to 0e08316 Compare November 14, 2022 19:20

Heenawter commented Nov 14, 2022

View reviewed changes

Heenawter marked this pull request as ready for review November 14, 2022 19:25

Heenawter requested a review from a team as a code owner November 14, 2022 19:25

kibanamachine added the Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas label Nov 14, 2022

Add + clean up functional tests

7c5f88c

Heenawter force-pushed the add-data-view-runtime-field-support_2022-11-09 branch from 0e08316 to 7c5f88c Compare November 14, 2022 19:26

nreese reviewed Nov 14, 2022

View reviewed changes

Rename function based on feedback

b1fa3e1

nreese approved these changes Nov 14, 2022

View reviewed changes

Heenawter enabled auto-merge (squash) November 14, 2022 22:49

Heenawter merged commit 3190edd into elastic:main Nov 14, 2022

Heenawter deleted the add-data-view-runtime-field-support_2022-11-09 branch December 30, 2022 18:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Dashboard] [Controls] Add unmapped runtime field support to options list #144947

[Dashboard] [Controls] Add unmapped runtime field support to options list #144947

Heenawter commented Nov 9, 2022 •

edited

Loading

Heenawter Nov 14, 2022

Heenawter Nov 14, 2022 •

edited

Loading

elasticmachine commented Nov 14, 2022

nreese Nov 14, 2022

Heenawter Nov 14, 2022 •

edited

Loading

nreese left a comment

kibana-ci commented Nov 14, 2022

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

[Dashboard] [Controls] Add unmapped runtime field support to options list #144947

[Dashboard] [Controls] Add unmapped runtime field support to options list #144947

Conversation

Heenawter commented Nov 9, 2022 • edited Loading

Summary

How to Test

Checklist

For maintainers

Heenawter Nov 14, 2022

Choose a reason for hiding this comment

Heenawter Nov 14, 2022 • edited Loading

Choose a reason for hiding this comment

elasticmachine commented Nov 14, 2022

nreese Nov 14, 2022

Choose a reason for hiding this comment

Heenawter Nov 14, 2022 • edited Loading

Choose a reason for hiding this comment

nreese left a comment

Choose a reason for hiding this comment

kibana-ci commented Nov 14, 2022

💚 Build Succeeded

Metrics [docs]

Async chunks

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

Heenawter commented Nov 9, 2022 •

edited

Loading

Heenawter Nov 14, 2022 •

edited

Loading

Heenawter Nov 14, 2022 •

edited

Loading