fix(presto): default unknown types to string type #10753
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
villebro
commented
Sep 1, 2020
Comment on lines
+177
to
+179
| def _create_column_info( | ||
| cls, name: str, data_type: types.TypeEngine | ||
| ) -> Dict[str, Any]: |
There was a problem hiding this comment.
Not sure why MyPy hadn't picked up on the calls sending TypeEngine to data_type, not str.
Codecov Report
@@ Coverage Diff @@
## master #10753 +/- ##
==========================================
- Coverage 64.98% 60.31% -4.67%
==========================================
Files 789 789
Lines 37179 37181 +2
Branches 3555 3555
==========================================
- Hits 24161 22426 -1735
- Misses 12911 14568 +1657
- Partials 107 187 +80
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
john-bodley
reviewed
Sep 1, 2020
| (see MSSQL for example of NCHAR/NVARCHAR handling). | ||
|
|
||
| :param type_: Column type returned by inspector | ||
| :return: SqlAlchemy column type | ||
| """ | ||
| if not type_: |
There was a problem hiding this comment.
Is this somewhat of an anti-pattern, i.e., passing type_ as None results in the short-circuit and returning None . Can we not call get_sqla_column_type if the type is None?
There was a problem hiding this comment.
I'm not sure what calling this with None should return. As far as I'm aware, only the Presto db engine spec currently does this, so it would be nice to get more context and tests (see #10743 (comment)) to make sure this is working as intended.
john-bodley
approved these changes
Sep 21, 2020
villebro
force-pushed
the
villebro/presto-null-type
branch
from
803337d
to
53d5ad2
Compare
dpgaspar
pushed a commit
that referenced
this pull request
Sep 22, 2020
* fix(presto): default unknown types to string type * lint
amitmiran137
pushed a commit
to ofekisr/incubator-superset
that referenced
this pull request
Sep 22, 2020
…l_access/dashboard_by_id_endpoints * upstream/master: (29 commits) fix(presto): default unknown types to string type (apache#10753) feat(row-level-security): add base filter type and filter grouping (apache#10946) docs: add gallery screenshot & link in README (apache#10988) docs: add a "Gallery" page (apache#10968) build: add PR lint action (apache#10990) adding filters back that caused issues (apache#10989) chore: selectors refactor in SQLLab test suite (Cypress) (apache#10944) ESLint: Remove ts-ignore comments (apache#10933) style: fix checkbox color (apache#10970) fix: changed disabled rules in datasets module (apache#10979) fix: update the time filter for 'Last Year' option in explore (apache#10829) fix: use nullpool even for user lookup in the celery (apache#10938) Allow empty observations in alerting (apache#10939) fix: re-enabling several globally disabled lint rules (apache#10957) fix: setting specific exceptions common/query_context.py (apache#10942) Pylint disabled rule `pointless-string-statement` is not raising warining anymore - removing (apache#10975) fix: pylint disabled rules in dashboard/api.py (apache#10976) fix: removed disabled lint rule `too-many-locals` in connectors/base/models.py (apache#10958) ESLint: Re-enable rule no-access-state-in-setstate (apache#10870) fix: simply is_adhoc_metric (apache#10964) ...
auxten
pushed a commit
to auxten/incubator-superset
that referenced
this pull request
Nov 20, 2020
* fix(presto): default unknown types to string type * lint
mistercrunch
added
🏷️ bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
A recent PR #10658 introduced a regression on Presto for undefined column types. The regression was caused due to one of the methods calling the
get_sqla_column_typemethod withNonewhich is contradictory with the method signature. This reverts exception raising for unknown types and defaults those tosqlalchemy.types.String(), while at the same time logging the discrepancy.Going forward the associated methods need to be broken up and properly unit tested, as they are currently very difficult to follow.
TEST PLAN
CI + added test
ADDITIONAL INFORMATION