feat(sql): index SQL files and Dataform sqlx models - #1757
Open
paran-hiberuscom wants to merge 6 commits into
Open
feat(sql): index SQL files and Dataform sqlx models#1757paran-hiberuscom wants to merge 6 commits into
paran-hiberuscom wants to merge 6 commits into
Conversation
Vendors the DerekStride/tree-sitter-sql grammar (ABI 15) to extract tables, views, functions, columns, function calls, and the tables a query reads from FROM/JOIN and INSERT/UPDATE/DELETE targets.
A .sqlx file becomes one model node (kind class, qualified schema.name from
its config), and every model it names through ref()/resolve() in a ${...} span
or a js block, or lists in the config dependencies, becomes a resolved
references edge so impact and explore walk the model DAG.
A hand scanner in the shape of MyBatisExtractor, not a grammar: a .sqlx is not
valid SQL until the Dataform blocks and interpolations are masked, and the
edges live in those spans rather than in the SQL tree. The masked remainder
still goes through the SQL extractor, so literal FROM/JOIN source tables are
referenced too.
Two Dataform idioms produced no edge. A pre_operations / post_operations
body is SQL, so the ${...} refs written inside it are collected like any
other; the block itself is still blanked from the SQL body.
A backtick quotes an identifier in BigQuery, not a string, so `${ref("x")}`
is a table name being interpolated. The SQL-body scan now reads through
backticks; single and double quotes remain strings that hide what they hold.
… a duplicate test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1756. Six commits, rebased on today's
main, no refactor dependencies.Files:
src/extraction/languages/sql.ts,src/extraction/sqlx-extractor.ts,src/extraction/wasm/tree-sitter-sql.wasm, registry andtypes.tswiring,tree-sitter-helpers.tsline index shared with the MyBatis extractor, tests inextraction.test.tsandsqlx-extractor.test.ts, README, CHANGELOG and the site languages table.Verified:
npx vitest run __tests__/extraction.test.ts __tests__/sqlx-extractor.test.ts: 667 pass.main, none added.scripts/add-lang/verify-extraction.mjson jOOQ/sakila: PASS, 34 files, 483 symbols, 768 edges.Not done: the three-repo agent A/B benchmark from the add-lang skill. Happy to run it if you want it before review.