feat(dashboard): Add Drill to Detail modal w/ chart menu + right-click support #20728

codyml · 2022-07-15T23:39:21Z

SUMMARY

This PR introduces a drill-to-detail modal. It adds a "Drill to detail" menu item to dashboard charts' contextual menus and also completes support for opening the modal by right-clicking on chart features. The drill-to-detail modal allows users to inspect the data that is backing a chart from the dashboard that the chart is on, while optionally applying filters based on where in the chart was clicked.

More details:

Unlike the Samples pane, the drill-to-detail modal content is server-paginated and allows paging through all rows of data.
All columns of the dataset are displayed.
Native dashboard filters, cross-filters, ad-hoc chart filters and time chart filters are transparently respected; only rows matching those filters are displayed in the modal table.
The chart menu item and contextual menus appear when the DRILL_TO_DETAIL feature flag is enabled.
Only ECharts (excluding Tree Chart) are currently supported for contextual menu access.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Menu item:

Modal:

Modal w/ native filters:

TESTING INSTRUCTIONS

Ensure no change to contextual menu without feature flag set
With feature flag set, compare expected datasource results to drill to detail results in the following circumstances:
- No filters from chart or dashboard context
- Chart time filters
- Chart ad-hoc filters
- Dashboard native filters
- Dashboard cross-filters
Ensure that paginated results in modal work as expected in the following circumstances:
- No results
- One page of results
- Multiple pages of results
- Few columns
- Lots of columns
Check that right-clicking parts of charts opens the appropriate filters
- See below for list of charts that are supported
- Try "Drill to detail", "Drill by X" and "Drill by all" options
- Try filtering by string values, number values, and date values
- Try clearing filters

Charts supporting right-click:

Big number
- Right-click on number: Drill to detail
Big number with Trendline
- Right-click on number: Drill to detail
- Right-click on point on trendline: Drill by {date}
Box Plot
- Right-click on box: Drill by {dimension}
Funnel Chart
- Right-click on funnel segment: Drill by {dimension}
Gauge Chart
- Right-click on gauge segment or needle: Drill by {dimension}
Graph Chart
- Right-click on edge: Drill by {dimension}
Mixed Time-Series
- Right-click on point on line: Drill by {dimension}, Drill by {timestamp}, Drill by all
Pie Chart
- Right-click on pie segment: Drill by {dimension}
Radar Chart
- Right-click on dimension point: Drill by {dimension}
Time-series Chart
- Time-series Line Chart
- Time-series Area Chart
- Time-series Bar Chart v2
- Time-series Scatter Plot
- Time-series Smooth Line
- Time-series Stepped Line
- For all of these: Right-click on point on line: Drill by {dimension}, Drill by {timestamp}, Drill by all
Treemap v2
- Right-click on dimension title (for multiple-dimension treemaps): Drill by {dimension}
- Right-click on box: Drill by {dimension}, Drill by {addl dimension}, Drill by all

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags: DRILL_TO_DETAIL (DASHBOARD_CROSS_FILTERS optional to test cross-filter support)
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

superset-frontend/src/dashboard/components/DatasourceResultsPane/index.tsx

codecov · 2022-07-22T00:22:03Z

Codecov Report

Merging #20728 (0bf4e56) into master (0bf4e56) will not change coverage.
The diff coverage is n/a.

❗ Current head 0bf4e56 differs from pull request most recent head 2310b03. Consider uploading reports for the commit 2310b03 to get more accurate results

@@           Coverage Diff           @@
##           master   #20728   +/-   ##
=======================================
  Coverage   66.16%   66.16%           
=======================================
  Files        1773     1773           
  Lines       67689    67689           
  Branches     7218     7218           
=======================================
  Hits        44786    44786           
  Misses      21059    21059           
  Partials     1844     1844

Flag	Coverage Δ
javascript	`52.12% <0.00%> (ø)`
mysql	`80.98% <0.00%> (ø)`
postgres	`81.04% <0.00%> (ø)`
python	`81.15% <0.00%> (ø)`
sqlite	`79.64% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

superset-frontend/packages/superset-ui-core/src/query/types/Query.ts

zhaoyongjie

just left some suggestions. codes look good. I will test it in local. Thanks @codyml

superset-frontend/src/dashboard/components/DatasourceResultsPane/index.tsx

superset-frontend/src/components/Chart/chartAction.js

superset-frontend/src/dashboard/components/DrillDetailPane/DrillDetailPane.tsx

zhaoyongjie · 2022-07-25T06:31:43Z

superset-frontend/src/dashboard/components/DrillDetailPane/DrillDetailPane.tsx

+  initialFilters?: BinaryQueryObjectFilterClause[];
+}) {
+  const theme = useTheme();
+  const [pageIndex, setPageIndex] = useState(0);


Suggested change

const [pageIndex, setPageIndex] = useState(0);

const [pageIndex, setPageIndex] = useState(1);

I'm using 0-indexed page indices in front-end code because the TableView component does and I felt like consistency with that was more important – that okay?

The TableView has a property initialPageIndex, this property means which page is the first page when pagination. BTW, in the pagination context, the index means page number instead of index, so maybe the first page is 1.

superset/superset-frontend/src/components/TableView/TableView.tsx

Lines 118 to 139 in e214e1a

const TableView = ({

columns,

data,

pageSize: initialPageSize,

totalCount = data.length,

initialPageIndex,

initialSortBy = [],

loading = false,

withPagination = true,

emptyWrapperType = EmptyWrapperType.Default,

noDataText,

showRowCount = true,

serverPagination = false,

columnsForWrapText,

onServerPagination = () => {},

...props

}: TableViewProps) => {

const initialState = {

pageSize: initialPageSize ?? DEFAULT_PAGE_SIZE,

pageIndex: initialPageIndex ?? 0,

sortBy: initialSortBy,

};

Oh great, didn't see that thanks! Will update.

@zhaoyongjie I'm having trouble using initialPageIndex to tell TableView to use 1-indexed page numbers: it seems like for server pagination, that parameter is used to set the current page number instead, and TableView expects the first page index to be 0. If you can figure out how to do it without rewriting TableView to expect 1-indexed page numbers, feel free to push a commit.

superset-frontend/src/dashboard/components/DrillDetailPane/index.ts

superset-frontend/src/dashboard/components/SliceHeaderControls/index.tsx

superset-frontend/src/dashboard/components/DrillDetailPane/DrillDetailPane.tsx

michael-s-molina · 2022-07-25T18:16:06Z

Hi @codyml. We need to account for charts and dashboard configurations when displaying the drilling details panel. One example is the chart below:

In this chart, we should consider the controls configuration that impacts the query like is_software_dev = 1, last_yr_income <= 200000, and time range = 2018-03-01 <= col < 2022-07-25. We also need to consider that each visualization has different controls.

I believe the samples pane in Explore should also account for these filters. If I set is_software_dev = 1 I don't expect to see samples where this condition is not true.

The drilling details panel should also be influenced by dashboard configurations like developer_type in the example below:

To make things more complicated, dashboard filters can come from the filters panel or cross-filters.

I don't know if I covered everything. We may have other sources that impact a chart. Tagging some folks to see if I'm missing anything @rusackas @lauderbaugh @zhaoyongjie @kgabryje @geido

zhaoyongjie · 2022-07-26T01:32:51Z

I believe the samples pane in Explore should also account for these filters. If I set is_software_dev = 1 I don't expect to see samples where this condition is not true.

It's not true. The SamplesPane implies sampling the data source and has no relation to the chart. V1 API is at here, legacy API is at here.

It is a point worth discussing why the original implementation did not reset the filters field in the QueryObject? IMO, It's a bug.

The filters should be applied where clause or having clause in the query. The metrics has been reset on the QueryObject, so 1) we did not have a chance to send a query with having clause. 2) the time range has been reset so that no chance to send a query with a time partition.

codyml · 2022-07-27T14:39:58Z

Closed to refocus on additional work required given clarified requirements.

codyml · 2022-07-29T13:52:15Z

Reopening!

superset/common/query_actions.py

michael-s-molina · 2022-07-29T14:37:32Z

Can we display the modal with a fixed height/width? I find it weird that it starts small and grows after the results are rendered.

michael-s-molina · 2022-08-22T16:53:37Z

All issues should now be fixed, with the exception of D2D on values containing commas, which is being worked on in separate PRs (#21151, #21124).

@codyml Both PRs are now merged in case you want to rebase.

jinghua-qa · 2022-08-22T17:29:26Z

Dashboard Regression Test LGTM

…les pane.

…date.

jinghua-qa

LGTM

github-actions · 2022-08-22T22:57:42Z

Ephemeral environment shutdown and build artifacts deleted.

JunTech · 2022-09-05T06:45:09Z

how to use in 2.0.0 branch

codyml · 2022-09-06T14:29:53Z

how to use in 2.0.0 branch

Hi @JunTech! This feature wasn't included in the 2.0.0 release, so using it would require running Superset from your own non-release branch that includes this commit (and previous supporting commits), then enabling the DRILL_TO_DETAIL feature flag. This feature is very much still a work-in-progress and you're likely to run into incomplete support for viz plugins and the potential for bugs when enabled. It'll be included in a release when it's more complete!

pull-request-size bot added the size/L label Jul 15, 2022

codyml force-pushed the codyml/sc-52488/apply-drill-to-detail-modal-filters-gathered branch from f663ec8 to 54a18c1 Compare July 18, 2022 22:13

pull-request-size bot added size/XL and removed size/L labels Jul 19, 2022

geido reviewed Jul 20, 2022

View reviewed changes

codyml force-pushed the codyml/sc-52488/apply-drill-to-detail-modal-filters-gathered branch from 091f022 to 12d3f56 Compare July 20, 2022 16:59

pull-request-size bot added size/XXL and removed size/XL labels Jul 20, 2022

zhaoyongjie mentioned this pull request Jul 21, 2022

feat: modal of drill to detail on the dashboard #20697

Closed

9 tasks

codyml changed the title ~~feature(dashboard): Drill to detail modal~~ [WIP] feature(dashboard): Drill to detail modal Jul 22, 2022

codyml marked this pull request as ready for review July 22, 2022 00:14

codyml force-pushed the codyml/sc-52488/apply-drill-to-detail-modal-filters-gathered branch from 3ca3f87 to 2e4f5ee Compare July 22, 2022 00:21

codyml force-pushed the codyml/sc-52488/apply-drill-to-detail-modal-filters-gathered branch from 2e4f5ee to 9b7e22c Compare July 22, 2022 15:16

pull-request-size bot added size/XL and removed size/XXL labels Jul 22, 2022

zhaoyongjie reviewed Jul 22, 2022

View reviewed changes

superset-frontend/packages/superset-ui-core/src/query/types/Query.ts Show resolved Hide resolved

zhaoyongjie reviewed Jul 22, 2022

View reviewed changes

zhaoyongjie self-requested a review July 25, 2022 06:37

zhaoyongjie mentioned this pull request Jul 25, 2022

feat: a simple LRUCache in frontend #20842

Merged

9 tasks

zhaoyongjie reviewed Jul 25, 2022

View reviewed changes

codyml closed this Jul 26, 2022

zhaoyongjie mentioned this pull request Jul 29, 2022

[WIP] Drill modal apply all of the filters #20912

Closed

9 tasks

codyml reopened this Jul 29, 2022

zhaoyongjie reviewed Jul 29, 2022

View reviewed changes

superset/common/query_actions.py Outdated Show resolved Hide resolved

codyml added 16 commits August 22, 2022 10:30

Add drill-to-detail modal.

681916d

Include additional filters from dashboard context in request.

f3e1159

Set page cache size to be approximately equal to memory usage of Samp…

b8444b3

…les pane.

Update getDatasourceSamples signature.

802851b

One-line import/export.

2546841

Fix incorrect argument order in getDatasourceSamples invocation.

314fa51

Fix height of modal.

cdae1b3

Disable option in chart menu unless feature flag is set.

cb5c51f

Open modal on right-click.

0229a14

Fix double requests on modal open, controls disappearing on filter up…

dcd69a4

…date.

Show formattedVal in clearable filter tag.

cde67a9

Set force=false for all requests.

9546d12

Rearrange/refactor DrillDetailPane.

d9a6b91

Reset page index on reload.

d842f0a

Fix endless re-requests on request failure.

dc975d0

Fix modal layout issues.

2310b03

codyml force-pushed the codyml/sc-52488/apply-drill-to-detail-modal-filters-gathered branch from 621ea73 to 2310b03 Compare August 22, 2022 17:31

jinghua-qa approved these changes Aug 22, 2022

View reviewed changes

jinghua-qa merged commit 52648ec into apache:master Aug 22, 2022

geido mentioned this pull request Aug 24, 2022

chore: E2E tests for the Drill to detail modal #21187

Merged

9 tasks

mistercrunch added the 🚢 2.1.3 label Feb 18, 2024

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 and removed 🚢 2.1.3 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(dashboard): Add Drill to Detail modal w/ chart menu + right-click support #20728

feat(dashboard): Add Drill to Detail modal w/ chart menu + right-click support #20728

codyml commented Jul 15, 2022 •

edited

codecov bot commented Jul 22, 2022 •

edited

zhaoyongjie left a comment

zhaoyongjie Jul 25, 2022 •

edited

codyml Jul 26, 2022

zhaoyongjie Aug 8, 2022

codyml Aug 8, 2022

codyml Aug 8, 2022

michael-s-molina commented Jul 25, 2022 •

edited

zhaoyongjie commented Jul 26, 2022 •

edited

codyml commented Jul 27, 2022

codyml commented Jul 29, 2022

michael-s-molina commented Jul 29, 2022

michael-s-molina commented Aug 22, 2022 •

edited

jinghua-qa commented Aug 22, 2022

jinghua-qa left a comment

github-actions bot commented Aug 22, 2022

JunTech commented Sep 5, 2022

codyml commented Sep 6, 2022

	const [pageIndex, setPageIndex] = useState(0);
	const [pageIndex, setPageIndex] = useState(1);

	const TableView = ({
	columns,
	data,
	pageSize: initialPageSize,
	totalCount = data.length,
	initialPageIndex,
	initialSortBy = [],
	loading = false,
	withPagination = true,
	emptyWrapperType = EmptyWrapperType.Default,
	noDataText,
	showRowCount = true,
	serverPagination = false,
	columnsForWrapText,
	onServerPagination = () => {},
	...props
	}: TableViewProps) => {
	const initialState = {
	pageSize: initialPageSize ?? DEFAULT_PAGE_SIZE,
	pageIndex: initialPageIndex ?? 0,
	sortBy: initialSortBy,
	};

feat(dashboard): Add Drill to Detail modal w/ chart menu + right-click support #20728

feat(dashboard): Add Drill to Detail modal w/ chart menu + right-click support #20728

Conversation

codyml commented Jul 15, 2022 • edited

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

Charts supporting right-click:

ADDITIONAL INFORMATION

codecov bot commented Jul 22, 2022 • edited

Codecov Report

zhaoyongjie left a comment

Choose a reason for hiding this comment

zhaoyongjie Jul 25, 2022 • edited

Choose a reason for hiding this comment

codyml Jul 26, 2022

Choose a reason for hiding this comment

zhaoyongjie Aug 8, 2022

Choose a reason for hiding this comment

codyml Aug 8, 2022

Choose a reason for hiding this comment

codyml Aug 8, 2022

Choose a reason for hiding this comment

michael-s-molina commented Jul 25, 2022 • edited

zhaoyongjie commented Jul 26, 2022 • edited

codyml commented Jul 27, 2022

codyml commented Jul 29, 2022

michael-s-molina commented Jul 29, 2022

michael-s-molina commented Aug 22, 2022 • edited

jinghua-qa commented Aug 22, 2022

jinghua-qa left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 22, 2022

JunTech commented Sep 5, 2022

codyml commented Sep 6, 2022

codyml commented Jul 15, 2022 •

edited

codecov bot commented Jul 22, 2022 •

edited

zhaoyongjie Jul 25, 2022 •

edited

michael-s-molina commented Jul 25, 2022 •

edited

zhaoyongjie commented Jul 26, 2022 •

edited

michael-s-molina commented Aug 22, 2022 •

edited