Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SQLParse when query has reserved word #7305

Merged
merged 2 commits into from
May 16, 2019
Merged

Conversation

betodealmeida
Copy link
Member

CATEGORY

Choose one

  • Bug Fix
  • Enhancement (new features, refinement)
  • Refactor
  • Add tests
  • Build / Development Environment
  • Documentation

SUMMARY

The following query cannot be parsed by sqlparse, even though it's valid:

WITH
    columns AS (SELECT metric FROM a),
    rows AS (SELECT metric FROM b)
SELECT
    c.metric AS m1,
    r.metric AS m2
FROM columns c
JOIN rows r

This happens because sqlparse treats rows as a reserved word, even though the query is valid in Presto.

Our users have been overcoming the problem by quoting rows. This prevents sqlparse from treating it as a reserved word. But it's confusing to users, because the underlying engine (Presto) doesn't require it.

Instead, I fixed it by following Postel's Law, and being liberal about user input.

TEST PLAN

I added a unit test that covers the bug, and another basic one.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

REVIEWERS

@codecov-io
Copy link

codecov-io commented Apr 15, 2019

Codecov Report

Merging #7305 into lyftga will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           lyftga    #7305      +/-   ##
==========================================
+ Coverage   64.58%   64.58%   +<.01%     
==========================================
  Files         425      425              
  Lines       20884    20886       +2     
  Branches     2297     2297              
==========================================
+ Hits        13488    13490       +2     
  Misses       7270     7270              
  Partials      126      126
Impacted Files Coverage Δ
superset/sql_parse.py 99.14% <100%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f1702b...028104d. Read the comment docs.

@betodealmeida
Copy link
Member Author

@betodealmeida
Copy link
Member Author

ping @mistercrunch

Copy link
Member

@mistercrunch mistercrunch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This code is very opaque (not what you added, it was already before this PR), but more tests is better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants