Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dataset): handle missing python_type gracefully #19553

Merged
merged 2 commits into from Apr 7, 2022

Conversation

villebro
Copy link
Member

@villebro villebro commented Apr 6, 2022

SUMMARY

In the new Dataset model, we check if a column is temporal by checking the python_type of all columns. However, the property isn't guaranteed to be implemented: https://github.com/sqlalchemy/sqlalchemy/blob/7935b76d9e5b5fd4e64b2c6c3473737186acf2db/lib/sqlalchemy/sql/type_api.py#L618-L635 . Therefore, if calling it raises NotImplementedError, we assume the column is not temporal.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue: closes Error while saving Virtual data set using explore 1.5.0rc1 #19546
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link
Member

@zhaoyongjie zhaoyongjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@villebro villebro force-pushed the villebro/fix-python-type-temporal branch from 77e0969 to b95fb1a Compare April 7, 2022 07:44
@codecov
Copy link

codecov bot commented Apr 7, 2022

Codecov Report

Merging #19553 (1508c66) into master (aa419b8) will decrease coverage by 12.91%.
The diff coverage is 42.85%.

❗ Current head 1508c66 differs from pull request most recent head b95fb1a. Consider uploading reports for the commit b95fb1a to get more accurate results

@@             Coverage Diff             @@
##           master   #19553       +/-   ##
===========================================
- Coverage   66.65%   53.74%   -12.92%     
===========================================
  Files        1680     1680               
  Lines       64280    64286        +6     
  Branches     6564     6564               
===========================================
- Hits        42849    34550     -8299     
- Misses      19729    28034     +8305     
  Partials     1702     1702               
Flag Coverage Δ
hive ?
mysql ?
postgres ?
presto 52.54% <42.85%> (-0.01%) ⬇️
python 56.10% <42.85%> (-26.29%) ⬇️
sqlite ?
unit 47.21% <42.85%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/connectors/sqla/utils.py 76.53% <42.85%> (-18.04%) ⬇️
superset/utils/dashboard_import_export.py 0.00% <0.00%> (-100.00%) ⬇️
superset/key_value/commands/upsert.py 0.00% <0.00%> (-89.59%) ⬇️
superset/key_value/commands/update.py 0.00% <0.00%> (-89.37%) ⬇️
superset/key_value/commands/delete.py 0.00% <0.00%> (-85.30%) ⬇️
superset/db_engines/hive.py 0.00% <0.00%> (-85.19%) ⬇️
superset/key_value/commands/delete_expired.py 0.00% <0.00%> (-80.77%) ⬇️
superset/dashboards/commands/importers/v0.py 14.79% <0.00%> (-75.15%) ⬇️
superset/datasets/commands/importers/v0.py 24.82% <0.00%> (-68.80%) ⬇️
superset/databases/commands/test_connection.py 31.42% <0.00%> (-68.58%) ⬇️
... and 266 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa419b8...b95fb1a. Read the comment docs.

@villebro villebro merged commit d9343a4 into apache:master Apr 7, 2022
@villebro villebro deleted the villebro/fix-python-type-temporal branch April 7, 2022 11:30
villebro added a commit that referenced this pull request Apr 8, 2022
* fix(dataset): handle missing python_type gracefully

* refactor TEMPORAL_TYPES

(cherry picked from commit d9343a4)
sadpandajoe pushed a commit to preset-io/superset that referenced this pull request Apr 12, 2022
* fix(dataset): handle missing python_type gracefully

* refactor TEMPORAL_TYPES

(cherry picked from commit d9343a4)
@sadpandajoe
Copy link
Contributor

🏷️ preset:2022.13

philipher29 pushed a commit to ValtechMobility/superset that referenced this pull request Jun 9, 2022
* fix(dataset): handle missing python_type gracefully

* refactor TEMPORAL_TYPES
@mistercrunch mistercrunch added 🍒 1.5.0 🍒 1.5.1 🍒 1.5.2 🍒 1.5.3 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.0.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels lts-v1 preset:2022.13 preset-io size/S 🍒 1.5.0 🍒 1.5.1 🍒 1.5.2 🍒 1.5.3 🚢 2.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error while saving Virtual data set using explore 1.5.0rc1
5 participants