[Lens] Random sampling feature #143221

dej611 · 2022-10-12T17:08:45Z

Summary

Closes #142958

This PR introduces the random sampling feature in Lens, as a layer setting.

Main features in the PR:

Add a new Layer Settings action/panel
Add a new Random sampling feature for each layer
Refactored DimensionContainer to be reusable across Lens

Todo list:

Some testing behaviour

Duplicating a layer should carry the current sampling value to the new layer
Dropping a field into the workspace when no layer is configured should not change the set sampling rate (set it before dropping the field)
Clearing layer should reset the sampling
When using Terms with Other option enabled check the Inspector that both requests are using the same seed

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

This was checked for breaking API changes and was labeled appropriately

dej611 · 2022-10-18T13:54:25Z

src/plugins/data/common/search/aggs/agg_configs.test.ts

@@ -16,6 +16,15 @@ import type { DataView } from '@kbn/data-views-plugin/common';
 import { stubIndexPattern } from '../../stubs';
 import { IEsSearchResponse } from '..';

+// Mute moment.tz warnings about not finding a mock timezone
+jest.mock('../utils', () => {


Debugging test failures was really hard with all warning from moment.tz. This fixed the issue assigning a fixed (supported) timezone for moment.

dej611 · 2022-10-18T13:56:13Z

src/plugins/data/common/search/aggs/utils/time_splits.ts

@@ -427,11 +427,11 @@ export function insertTimeShiftSplit(
  const timeRange = aggConfigs.timeRange;
  const filters: Record<string, unknown> = {};
  const timeField = aggConfigs.timeFields[0];
+  const timeFilter = getTime(aggConfigs.indexPattern, timeRange, {


This computation could be done once for all the timeShifts

elasticmachine · 2022-10-19T12:48:12Z

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

flash1293

Tested and everything works as expected, LGTM. Still think we should add some in-product documentation, but we can do that as a follow-up

MichaelMarcialis

Hey, @dej611! I've left you a few small comments for your review. As none of them are super critical, assuming you're able to address them, I'll go ahead and approve this now so I don't hold you up further.

Just for the sake of visibility, the plan is to follow this with another PR to address the following items that @dej611, @KOTungseth, and I have discussed:

Switch the slider's EuiFormRow from columnCompressed to rowCompressed to allow the slider to expand to the full available width of the flyout.
Show "Speed" and "Accuracy" labels at each of the slider's respective extremities to emphasize what is happening.
Change the slider's tick numbers to percentages for easier reading.
Update help text below the slider to briefly explaining what this feature does (supplied by @KOTungseth) and include a link to the full documentation.

x-pack/plugins/lens/public/datasources/form_based/layer_settings.tsx

.../plugins/lens/public/editor_frame_service/editor_frame/config_panel/dimension_container.scss

x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/flyout_container.scss

x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/flyout_container.tsx

…gs.tsx Co-authored-by: Michael Marcialis <michael@marcial.is>

…e/142958

kibana-ci · 2022-10-24T12:28:41Z

💚 Build Succeeded

Buildkite Build
Commit: 45eb2f9

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`data`	520	521	+1
`lens`	943	959	+16
total			+17

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`data`	2517	2523	+6
`lens`	576	581	+5
total			+11

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`lens`	1.3MB	1.3MB	+10.9KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`lens`	46	47	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`data`	437.6KB	439.0KB	+1.4KB
`lens`	29.2KB	29.2KB	+1.0B
total			+1.4KB

Unknown metric groups

API count

id	before	after	diff
`data`	3245	3251	+6
`lens`	669	674	+5
total			+11

History

💚 Build #81706 succeeded 165fe55
💔 Build #81494 failed f65125d
💔 Build #81388 failed bbc0fd2
💔 Build #81368 failed f9e1bb9
💛 Build #81055 was flaky 281b8c9
💚 Build #81000 succeeded 8131b5e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Dosant · 2022-10-24T13:07:03Z

src/plugins/data/common/search/aggs/agg_configs.ts

+  isSamplingEnabled() {
+    return (
+      isSamplingEnabled(this.opts.probability) &&
+      this.getRequestAggs().filter((agg) => !agg.type.hasNoDsl).length > 0


Trying to follow...
Could you please explain this condition?

this.getRequestAggs().filter((agg) => !agg.type.hasNoDsl).length > 0

That condition is set to detect a specific case when no sub-agg is available for the sampling, therefore it makes no sense to have it. That's the case of Count of Records in Lens, where it should just return the number of documents.

Dosant

App services code lgtm

* ✨ First pass with UI for random sampling * ✨ Initial working version * 🔥 Remove unused stuff * 🔧 Refactor layer settings panel * 🐛 Fix terms other query and some refactor * 🏷️ Fix types issues * 🐛 Fix sampling for other terms agg * 🐛 Fix issue with count operation * ✅ Fix jest tests * 🐛 fix test stability * 🐛 fix test with newer params * 💄 Add tech preview label * ✅ Add new tests for sampling * ✅ Add more tests * ✅ Add new test for suggestions * ✅ Add functional tests for layer actions and random sampling * Update x-pack/plugins/lens/public/datasources/form_based/layer_settings.tsx Co-authored-by: Michael Marcialis <michael@marcial.is> * 👌 Integrated design feedback Co-authored-by: Joe Reuter <johannes.reuter@elastic.co> Co-authored-by: Michael Marcialis <michael@marcial.is>

## Summary Adds the 8.8 documentation for the following: - Enable report sharing: #153429 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/reporting-getting-started.html - Random sampling feature: #143221 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/lens.html#improve-visualization-loading-time - Improve Ignore global filters UI: #154441 and #155280 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/lens.html#add-annotations --------- Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>

## Summary Adds the 8.8 documentation for the following: - Enable report sharing: elastic#153429 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/reporting-getting-started.html - Random sampling feature: elastic#143221 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/lens.html#improve-visualization-loading-time - Improve Ignore global filters UI: elastic#154441 and elastic#155280 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/lens.html#add-annotations --------- Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com> (cherry picked from commit 06a800f)

# Backport This will backport the following commits from `main` to `8.8`: - [[DOCS] Adds 8.8 Viz docs (#157215)](#157215)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>

## Summary Adds the 8.8 documentation for the following: - Enable report sharing: elastic#153429 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/reporting-getting-started.html - Random sampling feature: elastic#143221 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/lens.html#improve-visualization-loading-time - Improve Ignore global filters UI: elastic#154441 and elastic#155280 Docs preview: https://kibana_157215.docs-preview.app.elstc.co/guide/en/kibana/master/lens.html#add-annotations --------- Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>

dej611 added 3 commits October 12, 2022 10:28

✨ First pass with UI for random sampling

be740b6

Merge remote-tracking branch 'upstream/main' into feature/142958

dc16774

✨ Initial working version

b45c6de

dej611 added Team:Visualizations Visualization editors, elastic-charts and infrastructure Feature:Lens v8.6.0 labels Oct 12, 2022

dej611 added 14 commits October 13, 2022 17:41

🔥 Remove unused stuff

1448a22

🔧 Refactor layer settings panel

7d05809

🐛 Fix terms other query and some refactor

c341c02

Merge remote-tracking branch 'upstream/main' into feature/142958

284e729

🏷️ Fix types issues

1360949

Merge remote-tracking branch 'upstream/main' into feature/142958

234e4a8

🐛 Fix sampling for other terms agg

f7b9dd5

🐛 Fix issue with count operation

862931b

✅ Fix jest tests

5eb26e0

🐛 fix test stability

54c8fc8

🐛 fix test with newer params

3039c5d

💄 Add tech preview label

8966788

✅ Add new tests for sampling

a627ba0

Merge remote-tracking branch 'upstream/main' into feature/142958

8131b5e

dej611 commented Oct 18, 2022

View reviewed changes

dej611 added 4 commits October 18, 2022 16:42

✅ Add more tests

281b8c9

✅ Add new test for suggestions

13c8078

Merge remote-tracking branch 'upstream/main' into feature/142958

4f63f67

✅ Add functional tests for layer actions and random sampling

f9e1bb9

dej611 marked this pull request as ready for review October 19, 2022 12:48

dej611 requested review from a team as code owners October 19, 2022 12:48

dej611 added the release_note:enhancement label Oct 19, 2022

dej611 and others added 3 commits October 19, 2022 15:20

Merge branch 'main' into feature/142958

bbc0fd2

Merge branch 'main' into feature/142958

f65125d

Merge branch 'main' into feature/142958

165fe55

flash1293 approved these changes Oct 20, 2022

View reviewed changes

MichaelMarcialis approved these changes Oct 20, 2022

View reviewed changes

dej611 and others added 4 commits October 24, 2022 12:49

Merge remote-tracking branch 'upstream/main' into feature/142958

60a4580

Update x-pack/plugins/lens/public/datasources/form_based/layer_settin…

e71845e

…gs.tsx Co-authored-by: Michael Marcialis <michael@marcial.is>

Merge branch 'feature/142958' of github.com:dej611/kibana into featur…

c4a2b12

…e/142958

👌 Integrated design feedback

45eb2f9

Dosant reviewed Oct 24, 2022

View reviewed changes

Dosant approved these changes Oct 24, 2022

View reviewed changes

dej611 merged commit 9d819ba into elastic:main Oct 24, 2022

kibanamachine added the backport:skip This commit does not require backporting label Oct 24, 2022

dej611 deleted the feature/142958 branch October 25, 2022 07:58

dej611 mentioned this pull request Oct 25, 2022

[Lens] Revisit Random sampling UI #143929

Merged

9 tasks

stratoula added release_note:skip Skip the PR/issue when compiling release notes and removed release_note:enhancement labels Nov 14, 2022

ruflin mentioned this pull request Dec 2, 2022

Random sampler aggregation support in Lens #146770

Closed

KOTungseth mentioned this pull request Apr 25, 2023

[DOCS] Add documentation for 8.8 #155793

Closed

19 tasks

KOTungseth mentioned this pull request May 9, 2023

[DOCS] Adds 8.8 Viz docs #157215

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Lens] Random sampling feature #143221

[Lens] Random sampling feature #143221

dej611 commented Oct 12, 2022 •

edited

Loading

dej611 Oct 18, 2022

dej611 Oct 18, 2022

elasticmachine commented Oct 19, 2022

flash1293 left a comment

MichaelMarcialis left a comment

kibana-ci commented Oct 24, 2022

API count

Dosant Oct 24, 2022

dej611 Oct 24, 2022

Dosant left a comment •

edited

Loading

[Lens] Random sampling feature #143221

[Lens] Random sampling feature #143221

Conversation

dej611 commented Oct 12, 2022 • edited Loading

Summary

Checklist

Risk Matrix

For maintainers

dej611 Oct 18, 2022

Choose a reason for hiding this comment

dej611 Oct 18, 2022

Choose a reason for hiding this comment

elasticmachine commented Oct 19, 2022

flash1293 left a comment

Choose a reason for hiding this comment

MichaelMarcialis left a comment

Choose a reason for hiding this comment

kibana-ci commented Oct 24, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

Dosant Oct 24, 2022

Choose a reason for hiding this comment

dej611 Oct 24, 2022

Choose a reason for hiding this comment

Dosant left a comment • edited Loading

Choose a reason for hiding this comment

dej611 commented Oct 12, 2022 •

edited

Loading

Dosant left a comment •

edited

Loading