Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(druid): Explicitly cast col to TIMESTAMP #17101

Merged

Conversation

john-bodley
Copy link
Member

@john-bodley john-bodley commented Oct 13, 2021

SUMMARY

This speaks somewhat to SIP-15A and the proof-of-concept defined in #7682, but in theory any column in Druid (and not just __time which is of type TIMESTAMP) could be used as the temporal column and thus should first be converted to a TIMESTAMP.

This PR replicates the Presto logic which leveraging CAST(x AS TIMESTAMP) which works when x is a DATE, TIMESTAMP or STRING (these adhere to the ISO 8601 format per the column "DATETIME FORMAT"), i.e.,

druid> SELECT CAST(TIMESTAMP '2021-01-01 00:00:00' AS TIMESTAMP)
2021-01-01T00:00:00.000Z

druid> SELECT CAST(DATE '2021-01-01' AS TIMESTAMP)
2021-01-01T00:00:00.000Z

druid> SELECT CAST('2021-01' AS TIMESTAMP)
2021-01-01T00:00:00.000Z

Long term it would be good to implement something similar to the proof-of-concept to bring clarity to the various transforms and type conversions which are needed for both the left- and right-hand-side of a comparator.

TESTING INSTRUCTIONS

CI.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@john-bodley john-bodley force-pushed the john-bodley--druid-time-grains-cast branch from 81a9312 to 4c7c560 Compare October 13, 2021 21:49
@codecov
Copy link

codecov bot commented Oct 13, 2021

Codecov Report

Merging #17101 (625eaf0) into master (8d54dee) will decrease coverage by 0.03%.
The diff coverage is n/a.

❗ Current head 625eaf0 differs from pull request most recent head 4c7c560. Consider uploading reports for the commit 4c7c560 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17101      +/-   ##
==========================================
- Coverage   76.75%   76.72%   -0.04%     
==========================================
  Files        1031     1031              
  Lines       55198    55198              
  Branches     7504     7504              
==========================================
- Hits        42365    42348      -17     
- Misses      12583    12600      +17     
  Partials      250      250              
Flag Coverage Δ
hive ?
mysql 81.92% <ø> (?)
postgres 81.92% <ø> (?)
presto ?
python 82.01% <ø> (-0.06%) ⬇️
sqlite 81.61% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/druid.py 86.27% <ø> (ø)
superset/db_engines/hive.py 0.00% <0.00%> (-85.19%) ⬇️
superset/db_engine_specs/hive.py 69.76% <0.00%> (-17.06%) ⬇️
superset/db_engine_specs/presto.py 83.47% <0.00%> (-6.91%) ⬇️
superset/views/database/mixins.py 81.03% <0.00%> (-1.73%) ⬇️
superset/connectors/sqla/models.py 85.59% <0.00%> (-1.66%) ⬇️
superset/utils/core.py 89.84% <0.00%> (-0.13%) ⬇️
superset/views/base_api.py 98.31% <0.00%> (+0.42%) ⬆️
superset/common/query_context.py 90.90% <0.00%> (+0.45%) ⬆️
superset/security/manager.py 91.84% <0.00%> (+0.52%) ⬆️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d54dee...4c7c560. Read the comment docs.

@john-bodley john-bodley merged commit 565ee23 into apache:master Oct 18, 2021
opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021
Co-authored-by: John Bodley <john.bodley@airbnb.com>
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants