refactor(trino): Handful of updates for the Trino engine #20152
Conversation
Codecov Report
@@ Coverage Diff @@
## master #20152 +/- ##
==========================================
- Coverage 66.52% 66.38% -0.15%
==========================================
Files 1726 1726
Lines 64887 64845 -42
Branches 6832 6832
==========================================
- Hits 43169 43046 -123
- Misses 19986 20067 +81
Partials 1732 1732
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| "P1M": "date_trunc('month', CAST({col} AS TIMESTAMP))", | ||
| "P3M": "date_trunc('quarter', CAST({col} AS TIMESTAMP))", | ||
| "P1Y": "date_trunc('year', CAST({col} AS TIMESTAMP))", | ||
| # "1969-12-28T00:00:00Z/P1W", # Week starting Sunday |
There was a problem hiding this comment.
I'm really not sure why these were commented out. This has always been the case since the first version of this file.
There was a problem hiding this comment.
Trino was forked from Presto, I think comments this is safe. thanks for keeping clear up.
john-bodley
force-pushed
the
john-bodley--trino-db-engine-spec
branch
2 times, most recently
from
5925872
to
3aee525
Compare
john-bodley
force-pushed
the
john-bodley--trino-db-engine-spec
branch
from
3aee525
to
d7f2ed4
Compare
| @@ -18,7 +18,6 @@ | |||
| # | |||
| -r base.in | |||
| -e .[cors,druid,hive,mysql,postgres,thumbnails] | |||
| flask-cors>=2.0.0 | |||
There was a problem hiding this comment.
Duplicate logic per line #20.
| @@ -151,7 +150,7 @@ def get_git_sha() -> str: | |||
| "oracle": ["cx-Oracle>8.0.0, <8.1"], | |||
| "pinot": ["pinotdb>=0.3.3, <0.4"], | |||
| "postgres": ["psycopg2-binary==2.9.1"], | |||
| "presto": ["pyhive[presto]>=0.4.0"], | |||
| "presto": ["pyhive[presto]>=0.6.5"], | |||
There was a problem hiding this comment.
Bumping to ensure that PyHive doesn't specify the trino dialect.
|
Looks like this commit broke the trino integration. Error "trino error 'Cursor' object has no attribute 'poll'" was thrown with latest docker image. There is no error just if I use docker image just before this commit. |
|
@john-bodley, @zhaoyongjie guys. Trino engine using |
mistercrunch
added
🏷️ bot
SUMMARY
This PR provides a number of updates for the Trino engine including:
TrinoEngineSpecderive from thePrestoEngineSpecrather than theBaseEngineSpec. I'm not really sure as to why the later was used as it violates the DRY principle and also means that many of the Presto customizations—which Trino is forked from—were previously not defined.trinonativetotrino—which requires a PyHive bump—thanks to @betodealmeida's work chore: rename Trino entry point dropbox/PyHive#428. I thought there was merit going forward usingtrinorather thantrinonativeeverywhere.TrinoTemplateProcessorso macros like{{ trino.latest_partition(...) }}etc. can be used. Note made sure the processor also supported theprestokey for backwards compatibility given that it is likely people will be migrating from Presto and theprestokey is likely embedded in saved queries, virtual datasets, etc.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
CI.
ADDITIONAL INFORMATION