feat: Compile base & BigQuery's parser with mypyc#7206
Merged
Conversation
Contributor
SQLGlot Integration Test ResultsComparing:
By Dialect
Overallmain: 14755 total, 13152 passed (pass rate: 89.1%), sqlglot version: sqlglot:compile-parser-poc: 14755 total, 13181 passed (pass rate: 89.3%), sqlglot version: Transitions: |
Extract BigQuery parser into sqlglot/parsers/bigquery.py for separate compilation alongside parser.py. This enables mypyc to compile both modules together, with cross-module class attribute defaults resolved correctly via the mypyc globals export fix. Key changes: - Move BigQuery.Parser from dialects/bigquery.py to parsers/bigquery.py to allow mypyc compilation as a separate module - Expose module-level parser dicts (_FUNCTIONS, _NO_PAREN_FUNCTIONS, etc.) for cross-module reference by bigquery parser - Fix _NO_PAREN_FUNCTIONS: CURRENT_DATETIME was incorrectly mapped to CurrentDate instead of CurrentDatetime - Fix _pa helper in dialect metaclass to read mypyc compiled class attributes via temporary instance (getset descriptors aren't dicts) - Fix build_logarithm to read LOG_DEFAULTS_TO_LN from instance instead of walking __dict__ (mypyc descriptors aren't bools) - Add parsers/bigquery.py to sqlglotc/setup.py compilation targets - Refactor dialect metaclass to centralize parser attribute overrides that were previously duplicated across dialect Parser subclasses
90d6932 to
6788b61
Compare
VaggelisD
commented
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
reviewed
Mar 5, 2026
tobymao
approved these changes
Mar 5, 2026
georgesittas
reviewed
Mar 5, 2026
Collaborator
georgesittas
left a comment
There was a problem hiding this comment.
Getting an initial pass in before moving on to parser.py and parsers/bigquery.py
c377116 to
7b9e0cd
Compare
georgesittas
reviewed
Mar 6, 2026
Collaborator
georgesittas
left a comment
There was a problem hiding this comment.
Final set of comments from me, LGTM otherwise.
georgesittas
approved these changes
Mar 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR enables compilation for
parser.pyand as a PoC it also fully compiles BigQuery's parser. The point of this PR is to:ParserDialects, maintaining backwards compatibilityThe key changes include:
BigQuery.Parserinto its own file for compilation alongside parser.py (mypyc cannot deal with nested classes well)_FUNCTIONS,_NO_PAREN_FUNCTIONSetc.) as module-level variables so dialect subclasses can reference them without__dict__introspection (mypyc stores class attributes as getset descriptors instead of plain dicts, so__dict__walking doesn't work)_pahelper to handle mypyc-compiled class attributes by reading from a temporary instance instead of__dict__(same mypyc getset descriptor limitation)_NO_PAREN_FUNCTIONSbug:TokenType.CURRENT_DATETIMEwas mapped toexp.CurrentDateinstead ofexp.CurrentDatetime(pre-existing bug, exposed by the _pa fallback path triggered under mypyc)Sqlglot-mypy
sqlglot-mypy is a lightweight fork of mypy that carries mypyc bug fixes needed to compile sqlglot e.g things like cross-module class attribute defaults and globals export for separate compilation. It is only used at build time (via [build-system] requires + pip build isolation) and does not affect the main venv or type checking, which continue to use upstream mypy. This is intended to be used until all the bugs that we run into SQLGlot are fixed upstream, and as a testing ground for our own mypyc experiments.