Skip to content

feat: compile a query to warehouse SQL without executing it#30

Merged
jpetey75 merged 1 commit into
mainfrom
feat/compile-query
Jun 28, 2026
Merged

feat: compile a query to warehouse SQL without executing it#30
jpetey75 merged 1 commit into
mainfrom
feat/compile-query

Conversation

@jpetey75

Copy link
Copy Markdown
Contributor

Closes #5

What

Adds Query.compile() -> str — returns the warehouse SQL Lightdash would run for a query, without executing it or fetching any rows.

sql = (
    model.query()
    .metrics(model.metrics.revenue)
    .dimensions(model.dimensions.country)
    .filter(model.dimensions.status == "active")
    .compile()
)

Why

The original request: get compiled SQL to run elsewhere (the issue's bigframes example). It also complements #19 — the query API caps at query.maxLimit (100k), but .compile() gives you the SQL to run directly against your warehouse with no row cap, for bigframes / dbt / pipelines:

sql = model.query().metrics(model.metrics.revenue).limit(10_000_000).compile()
df = bigframes.pandas.read_gbq(sql)

How

  • Reuses the existing _build_payload(), POSTed to the v1 POST /projects/{uuid}/explores/{explore}/compileQuery endpoint (the same one the UI's "View SQL" uses), returning results.query.
  • One real gotcha, handled: the v1 endpoint is stricter than the v2 query endpoint — it requires additionalMetrics and an id on every filter group/rule (v2 injects ids server-side). _inject_filter_ids() adds them; no-filter queries send {} (the shape v1 accepts). The 422 that surfaced this is captured in the implementation comment.
  • No alias (to_sql() etc.) — idiomatic Python favours one name; .compile() matches Lightdash's own terminology, and it's documented in the SDK guide.

Verified

Live, end-to-end through the SDK — compile with a filter (exercising the id injection):

SELECT DATE_TRUNC(`pages`.date, MONTH) AS `pages_date_month`,
       COUNT(DISTINCT `pages`.page_id) AS `pages_page_view_count`
FROM `lightdash-analytics`.`prod`.`pages` AS `pages`
WHERE (( (DATE_TRUNC(`pages`.date, MONTH)) >= ('2025-01-01') ))
GROUP BY 1 ORDER BY `pages_date_month` DESC LIMIT 500

Tests

tests/test_compile.py (7, fake client): returns SQL, POSTs to the compile endpoint, additionalMetrics present, no-filter sends {}, filter groups/rules get ids, limit flows through unbounded, requires a client. Plus an env-gated live acceptance test (test_compile_query). Full unit suite green (138).

🤖 Generated with Claude Code

Add Query.compile() -> str, returning the SQL Lightdash would run for a query
without executing it or fetching rows. Useful for inspecting/debugging a query
or running it directly against the warehouse (bigframes, dbt, pipelines) when
you need more rows than the query API returns.

- Reuses the existing _build_payload(), POSTed to the v1
  `/explores/{explore}/compileQuery` endpoint, returning `results.query`.
- The v1 endpoint is stricter than the v2 query endpoint: it requires
  `additionalMetrics` and an `id` on every filter group/rule (v2 injects ids
  server-side). `_inject_filter_ids()` adds them; no-filter queries send `{}`.

Verified live end-to-end: compile with a filter returns correct SQL with a
WHERE clause. 7 unit tests (fake client) + an env-gated acceptance test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jpetey75 jpetey75 merged commit 472e0f6 into main Jun 28, 2026
2 checks passed
@jpetey75 jpetey75 deleted the feat/compile-query branch June 28, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable to get a compiled SQL query

1 participant