Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models #22853

Merged
merged 35 commits into from
Apr 10, 2023

Conversation

hughhhh
Copy link
Member

@hughhhh hughhhh commented Jan 25, 2023

SUMMARY

Cleaning up tech debt from intial SIP-68 work to move towards having newer models (Table, Dataset) to run in explore. Since we are deprioritizing that work, refactoring the ExploreMixin to power SqlaTable and Query which are legacy models. Main reason is to have one codepath on executing queries in explore vs. having divergent paths at the moment.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@rusackas
Copy link
Member

Pinging @villebro since I know he's keen on this. Might wanna fill out the PR description though, for the rest of the world ;)

@codecov
Copy link

codecov bot commented Jan 25, 2023

Codecov Report

Merging #22853 (548682e) into master (8ef49a5) will increase coverage by 0.23%.
The diff coverage is 68.22%.

❗ Current head 548682e differs from pull request most recent head dfe7191. Consider uploading reports for the commit dfe7191 to get more accurate results

@@            Coverage Diff             @@
##           master   #22853      +/-   ##
==========================================
+ Coverage   67.70%   67.94%   +0.23%     
==========================================
  Files        1918     1918              
  Lines       74133    73873     -260     
  Branches     8052     8052              
==========================================
- Hits        50193    50190       -3     
+ Misses      21887    21630     -257     
  Partials     2053     2053              
Flag Coverage Δ
hive 53.16% <54.49%> (+0.42%) ⬆️
javascript 53.89% <0.00%> (ø)
mysql 79.21% <68.78%> (+0.69%) ⬆️
postgres 79.29% <68.78%> (+0.69%) ⬆️
presto ?
python 83.03% <69.31%> (+0.59%) ⬆️
sqlite 77.78% <68.78%> (+0.68%) ⬆️
unit 53.02% <40.21%> (+0.38%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...d/src/SqlLab/components/SaveDatasetModal/index.tsx 52.27% <0.00%> (ø)
superset-frontend/src/SqlLab/fixtures.ts 100.00% <ø> (ø)
...src/dashboard/components/PropertiesModal/index.tsx 64.03% <ø> (ø)
...nents/controls/MetricControl/AdhocMetricOption.jsx 77.77% <0.00%> (ø)
superset/migrations/env.py 0.00% <0.00%> (ø)
superset/views/dashboard/mixin.py 95.00% <ø> (ø)
superset/models/sql_lab.py 77.86% <42.85%> (+2.86%) ⬆️
superset/connectors/sqla/models.py 86.32% <63.63%> (-3.29%) ⬇️
superset/utils/core.py 90.99% <66.66%> (+0.08%) ⬆️
superset/models/helpers.py 68.69% <69.74%> (+30.04%) ⬆️
... and 4 more

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@hughhhh hughhhh force-pushed the fix-explore-mixin branch 4 times, most recently from 36346e1 to 2b1414c Compare January 25, 2023 19:19
@hughhhh hughhhh changed the title chore: Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models chore(WIP): Refactor ExploreMixin to power both Datasets (SqlaTable) and Query models Jan 25, 2023
@hughhhh hughhhh force-pushed the fix-explore-mixin branch 8 times, most recently from 743a478 to 3d7ae3d Compare January 26, 2023 16:47
@hughhhh
Copy link
Member Author

hughhhh commented Jan 26, 2023

/test_env up

@hughhhh
Copy link
Member Author

hughhhh commented Jan 26, 2023

/testenv up

@github-actions
Copy link
Contributor

@hughhhh Ephemeral environment spinning up at http://34.221.152.250:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

@jinghua-qa
Copy link
Member

issue 1:
When power=query, missing data type in the left data panel in explore
Screen Shot 2023-01-27 at 8 45 56 AM
issue 2:
Column dropdown is empty in Metric when power=query

Screen.Recording.2023-01-27.at.8.46.46.AM.mov

issue 3:
Not able to remove metrics in explore when power = dataset

Screen.Recording.2023-01-27.at.8.52.24.AM.mov

@andrey-zayats
Copy link

andrey-zayats commented Jan 30, 2023

issue 4:
chart in explore returns a syntax error when the user applies time range (can't repro with ALL datasets).
on which I was able to repro (maybe they are broken):

  • video_game_sales
  • FCC 2018 Survey
  • birth_france_by_region
    image

issue 5:
aggregate field is not pre-filled in metric control when d&d any value from the list to the Metric section.
The issue is relevant for:

  • filters control (operator field should be pre-filled in with IN operator)
  • sort by control
aggregate.mp4

issue 6:

User is not able to create dataset when chart = power query.

create.dataset.mp4

Also user can't create chart using chart power query (the same error^).

save.chart.mp4

@hughhhh hughhhh merged commit a41f3eb into master Apr 10, 2023
45 of 47 checks passed
@github-actions
Copy link
Contributor

Ephemeral environment shutdown and build artifacts deleted.

preset-machine pushed a commit to preset-io/superset that referenced this pull request Apr 10, 2023
hughhhh added a commit that referenced this pull request Apr 10, 2023
jinghua-qa added a commit to preset-io/superset that referenced this pull request Apr 13, 2023
jinghua-qa added a commit to preset-io/superset that referenced this pull request Apr 14, 2023
sebastianliebscher pushed a commit to sebastianliebscher/superset that referenced this pull request Apr 28, 2023
sebastianliebscher pushed a commit to sebastianliebscher/superset that referenced this pull request Apr 28, 2023
timestamp = dttm_col.get_timestamp_expression(
time_grain=time_grain, template_processor=template_processor
)
timestamp = dttm_col.get_timestamp_expression(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hughhhh this logic causes an error if the dttm_col is associated with a query as opposed to a table as dttm_col which has been coerced to be of type TableColumn has no table associated with it, i.e., dttm_col.table is None and thus this throws an exception.

john-bodley added a commit to john-bodley/superset that referenced this pull request May 31, 2023
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.0.0 labels Mar 13, 2024
@mistercrunch mistercrunch deleted the fix-explore-mixin branch March 26, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset:2023.13 risk:breaking-change Issues or PRs that will introduce breaking changes size/XXL 🚢 3.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants