Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sql): add jinja support to metrics and expressions #15247

Merged
merged 2 commits into from
Jun 19, 2021

Conversation

villebro
Copy link
Member

@villebro villebro commented Jun 18, 2021

SUMMARY

This adds support for Jinja templating to calculated columns and SQL metrics. The PR also adds TypedDict types for adhoc metrics to improve code quality and linting.

BEFORE

Before Jinja was not processed in calculated columns nor SQL metrics
image
image

AFTER

Now Jinja in both calculated columns and SQL metrics are processed correctly:
image
image

TESTING INSTRUCTIONS

  1. Create a new chart
  2. Add a SQL metric with Jinja
  3. Add a calculated column to the dataset with Jinja
  4. Press "Run"
  5. See query execute successfully and notice that Jinja syntax in metric and and calculated column are rendered.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Jun 18, 2021

Codecov Report

Merging #15247 (68c75ed) into master (c7c6375) will decrease coverage by 0.22%.
The diff coverage is 100.00%.

❗ Current head 68c75ed differs from pull request most recent head 164785f. Consider uploading reports for the commit 164785f to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #15247      +/-   ##
==========================================
- Coverage   77.25%   77.03%   -0.23%     
==========================================
  Files         971      971              
  Lines       50308    50322      +14     
  Branches     6140     6140              
==========================================
- Hits        38867    38765     -102     
- Misses      11237    11353     +116     
  Partials      204      204              
Flag Coverage Δ
hive ?
mysql 81.71% <100.00%> (+<0.01%) ⬆️
postgres 81.72% <100.00%> (+<0.01%) ⬆️
presto ?
python 81.81% <100.00%> (-0.44%) ⬇️
sqlite 81.35% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/common/query_object.py 90.14% <100.00%> (ø)
superset/connectors/druid/models.py 82.79% <100.00%> (ø)
superset/connectors/sqla/models.py 88.22% <100.00%> (-1.59%) ⬇️
superset/typing.py 100.00% <100.00%> (ø)
superset/utils/core.py 88.93% <100.00%> (-0.13%) ⬇️
superset/viz.py 56.35% <100.00%> (ø)
superset/db_engines/hive.py 0.00% <0.00%> (-82.15%) ⬇️
superset/db_engine_specs/hive.py 69.20% <0.00%> (-17.21%) ⬇️
superset/db_engine_specs/presto.py 83.36% <0.00%> (-6.53%) ⬇️
superset/views/database/mixins.py 81.03% <0.00%> (-1.73%) ⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7c6375...164785f. Read the comment docs.

@pull-request-size pull-request-size bot added size/L and removed size/M labels Jun 18, 2021
Comment on lines -882 to +895
column_name = metric["column"].get("column_name")
column_name = cast(str, metric["column"].get("column_name"))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cast is necessary, as it's not possible to do proper union types where column_name is always defined if expression_type == 'SIMPLE' but None if expression_type == 'SQL'

Copy link
Member

@suddjian suddjian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@villebro villebro merged commit 822eb2e into apache:master Jun 19, 2021
@villebro villebro deleted the villebro/jinja-expand branch June 19, 2021 05:29
cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
* feat(sql): add jinja support to metrics and expressions

* add test
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
* feat(sql): add jinja support to metrics and expressions

* add test
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
* feat(sql): add jinja support to metrics and expressions

* add test
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset-io size/L 🚢 1.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants