Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(native-filters): Hide non-numeric columns in numeric range filter #15385

Merged
merged 6 commits into from Jun 27, 2021

Conversation

kgabryje
Copy link
Member

@kgabryje kgabryje commented Jun 25, 2021

SUMMARY

When user selects "Numerical range" native filter, only numeric columns should be available to choose. If there are no numeric columns in the dataset, show "No columns found" message in select column dropdown. When user has selected a non-numeric column and then changes filter type to numerical range, the column field should be reset.
We use type_generic field to check column's type. If a column doesn't have type_generic field, we assume that it's numeric as a failsafe.

Thank you @dpgaspar and @villebro for providing the backend code!

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Screen.Recording.2021-06-25.at.12.21.23.mov

TESTING INSTRUCTIONS

  1. Enable DASHBOARD_NATIVE_FILTERS feature flag
  2. Create a native filter and select "Numerical range" filter
  3. Verify that only numeric columns are available to choose from (exception: some datasets from test data, like message_channels, have columns with type_generic field unset. In those cases, we display them as a failsafe)
  4. Change filter type to Value and verify that all columns are available, not only numeric
  5. Select some non-numeric column and change back to numerical range - column field should be reset.

ADDITIONAL INFORMATION

  • Has associated issue: closes [native-filters] hide non-numeric columns in numeric range filter #15226
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments

@@ -103,6 +128,7 @@ export function ColumnSelect({
onChange={onChange}
options={options}
placeholder={t('Select a column')}
notFoundContent={t('No columns found')}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make this message display the currentFilterType when set, just in case? Something like "No numerical columns found" if filtering for numerical columns etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with No compatible columns found, adding currentFilterType would require an ugly name mapping


const FILTERS_FIELD_NAME = 'filters';

export const FILTER_GROUPS = {
TIME: ['filter_time', 'filter_timegrain', 'filter_timecolumn'],
Copy link
Member

@villebro villebro Jun 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: would TEMPORAL be more accurate? Also, would it be a good idea to list the supported datatypes instead?

{
  filter_time: [GenericDataType.TEMPORAL],
  filter_timegrain: [GenericDataType.TEMPORAL],
  filter_timecolumn: [GenericDataType.TEMPORAL],
  filter_select: [GenericDataType.STRING, GenericDataType.NUMERIC, GenericDataType.TEMPORAL],
  filter_range: [GenericDataType.NUMERIC],
}

or similar.

edit: long-term this info should be moved into the plugin metadata

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic suggestion, thank you!

@codecov
Copy link

codecov bot commented Jun 25, 2021

Codecov Report

Merging #15385 (983174a) into master (53aa351) will decrease coverage by 0.16%.
The diff coverage is 43.85%.

❗ Current head 983174a differs from pull request most recent head 4fd2b4e. Consider uploading reports for the commit 4fd2b4e to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #15385      +/-   ##
==========================================
- Coverage   77.26%   77.10%   -0.17%     
==========================================
  Files         975      975              
  Lines       50579    50610      +31     
  Branches     6204     6212       +8     
==========================================
- Hits        39081    39021      -60     
- Misses      11291    11382      +91     
  Partials      207      207              
Flag Coverage Δ
hive ?
javascript 71.76% <43.19%> (-0.05%) ⬇️
mysql 81.71% <100.00%> (+<0.01%) ⬆️
postgres 81.73% <100.00%> (+<0.01%) ⬆️
presto 81.43% <100.00%> (-0.01%) ⬇️
python 81.99% <100.00%> (-0.27%) ⬇️
sqlite 81.35% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...rset-frontend/src/components/ImportModal/index.tsx 69.89% <ø> (+0.10%) ⬆️
superset-frontend/src/components/Select/Select.tsx 0.00% <0.00%> (ø)
...src/dashboard/components/DashboardBuilder/utils.ts 84.61% <0.00%> (+6.04%) ⬆️
...src/dashboard/components/PropertiesModal/index.jsx 85.61% <ø> (ø)
...src/dashboard/components/gridComponents/Column.jsx 87.87% <ø> (ø)
...nd/src/dashboard/components/gridComponents/Row.jsx 86.66% <ø> (ø)
...tersConfigModal/FiltersConfigForm/DefaultValue.tsx 25.00% <ø> (+4.16%) ⬆️
superset/db_engine_specs/base.py 88.37% <ø> (ø)
superset/db_engine_specs/druid.py 88.63% <ø> (ø)
...onfigModal/FiltersConfigForm/FiltersConfigForm.tsx 69.52% <50.00%> (-0.65%) ⬇️
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53aa351...4fd2b4e. Read the comment docs.

Copy link
Member

@michael-s-molina michael-s-molina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Only one non-blocking suggestion.

I really liked the improvements 👏🏼

.filter(filterValues)
.map((col: Column) => col.column_name)
.sort((a: string, b: string) => a.localeCompare(b))
.map((column: any) => ({ label: column, value: column })),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.map((column: any) => ({ label: column, value: column })),
.map((column: string) => ({ label: column, value: column })),

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@amitmiran137 amitmiran137 merged commit 09c44d0 into apache:master Jun 27, 2021
mvoitko pushed a commit to mvoitko/superset that referenced this pull request Jun 28, 2021
apache#15385)

* feat(native-filters): Hide non-numeric columns in numeric range filter

* Return true if type_generic undefined

* Code review comments

* Replace any with string

* fix tests

* add missing columns to select

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
apache#15385)

* feat(native-filters): Hide non-numeric columns in numeric range filter

* Return true if type_generic undefined

* Code review comments

* Replace any with string

* fix tests

* add missing columns to select

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
apache#15385)

* feat(native-filters): Hide non-numeric columns in numeric range filter

* Return true if type_generic undefined

* Code review comments

* Replace any with string

* fix tests

* add missing columns to select

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
apache#15385)

* feat(native-filters): Hide non-numeric columns in numeric range filter

* Return true if type_generic undefined

* Code review comments

* Replace any with string

* fix tests

* add missing columns to select

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L v1.3 🚢 1.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[native-filters] hide non-numeric columns in numeric range filter
5 participants