[SIP-25] Proposal for text based dashboard filters

[stswn]

[SIP-25] Proposal for text based dashboard filters

Motivation

In our projects analysts need to browse through large dataset (7M rows). They often need to search for rows by values which are unique or have very large number of unique values. Current implementation of dashboard filters is based on limited number of value buckets and does not allow them to do this.

Proposed Change

Add new visualization type - 'Search Box' which would allow to specify filter values by free typing.

New chart type should at least enable filtering with '==' operator. Additionally every defined filter could allow for different filtering operators to be used:

For text columns:

• contains => column LIKE '%value%'
• does not contain => column NOT LIKE '%value%'
• starts with => column LIKE '%value'
• ends with => column LIKE 'value%'
• is empty => column IS NULL
• is not empty => column IS NOT NULL

For number columns: arithmetical comparison operators.

Similar functionality is already present in chart query configuration section. We would like to see similar functionality on dashboard level available to the end users of the dashboards.

New visualization would not need to perform any queries to render. It would be based on table metadata only.

New or Changed Public Interfaces

Base version of this improvement containing only ability to filter by exact value(==) requires only new visualization to be prepared. It would emit filters in the same way that 'Filter Box' and 'Table' charts do.

Extended version requires changes in how filter state is stored on Dashboard level - it should contain information about used comparison operator. Also new addFilter action should be added and getEffectiveExtraFilters function should be extended.

New dependencies

None

Migration Plan and Compatibility

n/a

Rejected Alternatives

Existing 'Filter Box' visualization could be extended to allow for free input, but it is probably better to separate slice which performs query from one which can be rendered without fetching any data.

[issue-label-bot]

Issue-Label Bot is automatically applying the label #enhancement to this issue, with a confidence of 0.96. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[stswn]

I would like to prepare a sample WIP PR for this. Should code for new search/filter visualization be placed in superset-ui-plugins or here?

[betodealmeida]

@Swian did you consider doing this as an native dashboard component, instead of a new viz type?

[stswn]

@betodealmeida I was thinking about it but I wanted to narrow the scope of this SIP.
I have seen #5269 and #5925 and this efforts to add native dashboard components seemed abandoned... If there is a consensus that filters should be native components in the future, I can try to prepare this as a component instead of chart.

[betodealmeida]

@Swian my understanding is that having filters as native components was the direction we wanted to go. Having them as visualization types has always been a hack that works well for most of the common cases, but the abstraction breaks when you try to push it further.

@mistercrunch @graceguo-supercat do you have any thoughts on this?

[jackyq2015]

@Swian Any progress on this?

Perhaps this "text based filter" can be merged into the current "filter box" since their similarity. The main difference is that the former filter will be applied to wide like operation but the later filter is only applied to an enumeration which is supposed to have limited candidate.

[rusackas]

This is covered by the Dashboard Native Filters feature released with 1.0, which can be enabled by feature flag. Please let me know if I'm misunderstanding any intent. If so, we can re-open, update some context here, and take it to a VOTE if needed. Thank you!

[rusackas]

Reopening since Native Filters doesn't quite cover what this SIP asks for. Technically this is discussion, so we'll leave it there.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[claytongulick]

Is there an existing solution for performing LIKE queries for searching inside of a text column? Got to this issue from google trying to find a solution.

[adamkarminski]

I'd also like to upvote this issue. Lack of a text box filter is very limiting for us as well.

[rusackas]

@stswn can you re-formulate this proposal for the newer world of Dashboard Native Filters, and re-open the [DISCUSS] thread on the dev@ list, please? We'd like to move it forward if there's any intention of following through here.

[stswn]

@rusackas I am no longer using Superset, and I am not familiar with current state of affairs. I am no longer interested in this and I won't be able to add required input. If no one else is interested in updating this SIP and taking it forward it should probably be closed.

[rusackas]

Closing as requested by the author. If anyone else is interested in taking this under their wing, let's re-open as a new SIP! Thanks all!

[andro42]

How can we reopen this SIP? I'd also like to have textbox type filter. It could be implemented as Superset Feature Flag.

[nkdanta1]

any news on this ?
we want to put a contain filter in the dashboards.

[rusackas]

@andro42 @nkdanta1 a SIP is generally opened by the person who intends to initiate/complete/sponsor/organize the work. If either of you would like to do so, I would suggest starting a new SIP rather than rekindling the old one. You can certainly copy/paste the relevant bits, and go into more technical detail abut building/testing the feature, and performance/architectural tradeoffs. We'd most certainly welcome this effort if you're into it.

[rusackas]

There is, however, a newer sip that's related. Perhaps some of the concepts in this SIP should be brought up in that discussion: #27666 (comment)