Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: generate consistent QueryObject whether GenericAxis is enabled or disabled #21519

Merged
merged 6 commits into from
Sep 21, 2022

Conversation

zhaoyongjie
Copy link
Member

@zhaoyongjie zhaoyongjie commented Sep 19, 2022

SUMMARY

This PR intends to generate a consistent QueryObject whether GenericAxis FF is enabled or disabled. e.g.: the charts wouldn't change on the Dashboard although Axis FF from enabled to disabled and vice versa.

  1. Make sure that the charts on the Dashboards are consistent whether FF is enabled or disabled, so the FF GenericAxis should be safety enabled by default (will do in separated PR).
  2. Add unit test in Echart timeseries charts.
  3. Add unit test in Echart MixedTimeseries charts.
  4. Move getXAxis and isXAxisSet into superset-ui/core

There are 2 kinds of queryObject patterns after this PR:

The QueryObjec will be like below when the x_axis is not in formData whether GenericAxis is enabled or disabled

{
        granularity: 'time_column',
        extras: { time_grain_sqla: 'P1Y', having: '', where: '' },
        is_timeseries: true,
        post_processing: [
          {
            operation: 'pivot',
            options: {
              aggregates: { 'count(*)': { operator: 'mean' } },
              columns: ['col1'],
              drop_missing_columns: true,
              index: ['__timestamp'],
            },
          },
          { operation: 'flatten' },
        ],
      }
     ....
     ....
}

The QueryObjec will be like below when the x_axis is in formData whether GenericAxis is enabled or disabled.

The granularity(axis) and time_grain will be converted to BASE_AXIS, then remove time_grain_sqla and is_timeseries

{
        extras: { having: '', where: '' },
        columns: [
          {
            columnType: 'BASE_AXIS',
            expressionType: 'SQL',
            label: 'time_column',
            sqlExpression: 'time_column',
            timeGrain: 'P1Y',
          },
          'col1',
        ],
        post_processing: [
          {
            operation: 'pivot',
            options: {
              aggregates: { 'count(*)': { operator: 'mean' } },
              columns: ['col1'],
              drop_missing_columns: true,
              index: ['time_column'],
            },
          },
          { operation: 'flatten' },
        ],
      }
     ....
     ....

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

Test Case 1

  1. Turn off GenericAxis
  2. Make an Echart time-series chart and a Mixed time-series, save to a new Dashboard.
  3. Turn on GenericAxis
  4. Refresh Dashboard
  5. Charts should be as before

Test Case 2

  1. Turn on GenericAxis
  2. Make an Echart time-series chart and a Mixed time-series, save to a new Dashboard.
  3. Turn off GenericAxis
  4. Refresh Dashboard
  5. Charts should be as before

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces a new feature or API
  • Removes existing feature or API

@pull-request-size pull-request-size bot added size/L and removed size/M labels Sep 20, 2022
Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesomen work, looks great! One minor comment, I'll test this later tonight EET.

@codecov
Copy link

codecov bot commented Sep 20, 2022

Codecov Report

Merging #21519 (cf11bf3) into master (fdb4702) will decrease coverage by 0.00%.
The diff coverage is 95.00%.

@@            Coverage Diff             @@
##           master   #21519      +/-   ##
==========================================
- Coverage   66.67%   66.66%   -0.01%     
==========================================
  Files        1793     1793              
  Lines       68493    68506      +13     
  Branches     7275     7279       +4     
==========================================
+ Hits        45665    45672       +7     
- Misses      20966    20971       +5     
- Partials     1862     1863       +1     
Flag Coverage Δ
javascript 52.83% <95.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...hart-echarts/src/MixedTimeseries/transformProps.ts 0.00% <0.00%> (ø)
...t-ui-chart-controls/src/operators/pivotOperator.ts 100.00% <100.00%> (ø)
...ui-chart-controls/src/operators/prophetOperator.ts 100.00% <100.00%> (ø)
...-ui-chart-controls/src/operators/renameOperator.ts 100.00% <100.00%> (ø)
...controls/src/operators/timeComparePivotOperator.ts 100.00% <100.00%> (ø)
...es/superset-ui-core/src/query/buildQueryContext.ts 100.00% <100.00%> (ø)
...nd/packages/superset-ui-core/src/query/getXAxis.ts 100.00% <100.00%> (ø)
.../superset-ui-core/src/query/normalizeTimeColumn.ts 100.00% <100.00%> (ø)
...in-chart-echarts/src/MixedTimeseries/buildQuery.ts 100.00% <100.00%> (ø)
.../plugin-chart-echarts/src/Timeseries/buildQuery.ts 71.42% <100.00%> (+4.76%) ⬆️
... and 4 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great work @zhaoyongjie, thanks for all these improvements! While testing I looked at the backend logs, and noticed one last deprecation notice:

2022-09-20 20:32:39,010:WARNING:superset.common.query_object:The field `timeseries_limit` is deprecated, please use `series_limit` instead.

This PR is already pretty big, so no need to fix this here, but once we replace timeseries_limit with series_limit we should be rid of all deprecation notices for the main ECharts viz types 👍

@zhaoyongjie
Copy link
Member Author

This is really great work @zhaoyongjie, thanks for all these improvements! While testing I looked at the backend logs, and noticed one last deprecation notice:

2022-09-20 20:32:39,010:WARNING:superset.common.query_object:The field `timeseries_limit` is deprecated, please use `series_limit` instead.

This PR is already pretty big, so no need to fix this here, but once we replace timeseries_limit with series_limit we should be rid of all deprecation notices for the main ECharts viz types 👍

Thanks @villebro. I will fix it in a separated PR.

@zhaoyongjie zhaoyongjie merged commit 4d12e37 into apache:master Sep 21, 2022
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 and removed 🚢 2.1.3 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XL 🚢 2.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants