Skip to content
This repository has been archived by the owner on Dec 10, 2021. It is now read-only.

feat: add control grouping functionality #485

Merged
merged 13 commits into from
May 20, 2020

Conversation

villebro
Copy link
Contributor

@villebro villebro commented May 13, 2020

🏆 Enhancements
This adds support for control value grouping introduced in apache/superset#9740 . If a control has specified the property queryFields, those are used to group control values together when the query is build. Example formData:

{
  "series": ["gender", "nationality"],
  "primary_metric": "age",
  "secondary_metric": "height",
  "queryFields": {
    "primary_metric": "metrics",
    "secondary_metric": "metrics",
    "series': 'groupby"
  }
}

This will result in the following query object:

{
  "metrics": ["primary_metric", "secondary_metric"],
  "groupby": ["gender", "nationality']
}

This will help decouple superset-ui from individual plugins, as plugins can specify an arbitrary number of metric, secondary_metric, series, groupby without there being a need for superset-ui to know which is which. This will also bring the query logic closer to the control panel where controls are defined, as the queryField can be defined in the control metadata, hence won't need to be mapped over in buildQuery.{js,ts}.

@vercel
Copy link

vercel bot commented May 13, 2020

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/superset/superset-ui/ognt2go4k
✅ Preview: https://superset-ui-git-fork-preset-io-villebro-autogroup-controls.superset.now.sh

@codecov
Copy link

codecov bot commented May 13, 2020

Codecov Report

Merging #485 into master will decrease coverage by 0.11%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #485      +/-   ##
==========================================
- Coverage   22.75%   22.63%   -0.12%     
==========================================
  Files         276      281       +5     
  Lines        6672     6692      +20     
  Branches      645      648       +3     
==========================================
- Hits         1518     1515       -3     
- Misses       5115     5138      +23     
  Partials       39       39              
Impacted Files Coverage Δ
.../superset-ui-control-utils/src/shared-controls.tsx 19.40% <ø> (ø)
...kages/superset-ui-query/src/types/QueryFormData.ts 100.00% <ø> (ø)
...s/plugin-chart-word-cloud/src/plugin/buildQuery.ts 100.00% <ø> (ø)
packages/superset-ui-query/src/buildQueryObject.ts 100.00% <100.00%> (ø)
packages/superset-ui-query/src/convertMetric.ts 100.00% <100.00%> (ø)
...ckages/superset-ui-query/src/extractQueryFields.ts 100.00% <100.00%> (ø)
packages/superset-ui-query/src/processGroupby.ts 100.00% <100.00%> (ø)
plugins/legacy-plugin-chart-table/src/index.ts 0.00% <0.00%> (ø)
plugins/legacy-plugin-chart-iframe/src/index.js 0.00% <0.00%> (ø)
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b608f0...511e3e7. Read the comment docs.

Copy link
Contributor

@kristw kristw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code details looks good. Only a few minor/refactor comments.

I hope you could describe a bit more in the PR description or code comment what does the
ResidualXXX mean? In particular, what was the behavior before and after.

packages/superset-ui-query/src/buildGroupedControls.ts Outdated Show resolved Hide resolved
packages/superset-ui-query/src/buildQueryObject.ts Outdated Show resolved Hide resolved
packages/superset-ui-query/src/processMetrics.ts Outdated Show resolved Hide resolved
@villebro
Copy link
Contributor Author

Thanks for the review @kristw ; my apologies for the brief description, I meant to expand on the initial description but got sidetracked. Will add more context to describe why this change is necessary and pointers for some design decisions. Also, as this is my first real stab at TS, I fully expect there to be inconsistencies, so if something looks weird, it's probably because I've done it wrong.

@ktmud
Copy link
Contributor

ktmud commented May 14, 2020

I think the term "control group" is kind of confusing. It took me quite a while to understand what problem this PR tries to solve. The term control group is too commonly used in Data Science in the context of hypothesis testing, i.e., A/B tests and I just couldn't get over it. At some point, I also thought this was about arranging the UI as "control" seems to indicate the tangible form control element.

If I'm getting it right, this is actually about consolidating control fields required for constructing DB queries (metrics, groupby, and columns). They may have arbitrary field names, but the backend needs to know which are metrics fields, which are groupby, etc.

It's not grouped form controls as in UI components, but more like grouped query API parameters or consolidated query fields. Can we maybe rename the variables/functions as such:

controlGroups -> queryFields
buildGroupedControls -> extractQueryFields
defaultedControlGroups -> aliases or queryFieldAliases

>;
export type ResidualFormData = {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
[key: string]: any;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto on unknown

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, are the items in ResidualFormData a QueryFormDataMetric? buildGroupedControlData seems to imply that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was also puzzled by this. ResidualFormData (in retrospect should be QueryFormResidualData) is currently expected to contain either column names (string) or metrics (QueryFormDataMetric which is the same as string | AdhocMetric). When these are combined, the union is reduced to string | AdhocMetric which is the same as QueryFormDataMetric. I'm tempted to remove QueryFormDataMetric and just pass around QueryFormResidualData, which can later be split into QueryFormDataMetric and QueryFormDataColumn when that need arises (Down the line we'll need to add AdhocColumn which is a column with a SQL expression). I will actually do that, we'll see how it looks.

const [key, residualValue] = entry;
if (Object.prototype.hasOwnProperty.call(defaultedControlGroups, key)) {
const controlGroup: string = defaultedControlGroups[key];
const controlValue = residualFormData[key];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused by what this function is doing. Aren't controlValue and residualValue the same thing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's what I think this function is essentially doing, correct me if I'm wrong?

const normalizedKey = Object.prototype.hasOwnProperty.call(defaultedControlGroups, key)
      ? defaultedControlGroups[key]
      : key;
 groupedControls[normalizedKey] = (groupedControls[normalizedKey] || []).concat(residualValue);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I, too, was a bit puzzled here, so I've tried to write a simplified version of this block as well. Curious if this is missing something important (I'm a little tired, so... probably):

Object.keys(residualFormData).forEach(key => {
  const conformedKey = defaultedControlGroups[key] || key;
  groupedControls[conformedKey] = [...groupedControls[conformedKey] || [], residualFormData[key]]
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're both right, this was poorly written 🤦 Will address.

@villebro
Copy link
Contributor Author

Thanks all for the comments! ❤️ Will implement changes and update description + comment where necessary.

Copy link
Member

@rusackas rusackas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vercel vercel bot temporarily deployed to Preview May 20, 2020 17:15 Inactive
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants