Skip to content

Optimize DJ SQL and fix Trino compatibility#1711

Merged
shangyian merged 3 commits intoDataJunction:mainfrom
shangyian:djsql-speed
Jan 26, 2026
Merged

Optimize DJ SQL and fix Trino compatibility#1711
shangyian merged 3 commits intoDataJunction:mainfrom
shangyian:djsql-speed

Conversation

@shangyian
Copy link
Copy Markdown
Collaborator

@shangyian shangyian commented Jan 25, 2026

Summary

  1. Optimizes /djsql/ endpoint by removing redundant validation. Validation is now delegated to build_metrics_sql(), avoiding duplicate work.
  2. Made dialect parameter optional in both /djsql/ endpoint and build_metrics_sql() rather than defaulting to Spark when not specified. DJ will try to infer an appropriate dialect based on the catalogs of the available tables.
  3. Fixes Trino compatibility issue with single-argument COALESCE (Trino requires COALESCE to have at least 2 arguments), but we can detect and only wrap columns in COALESCE when there are multiple CTEs.
  4. Added filter_dimensions to track dimensions referenced only in where clauses. These dimensions are joined for filter resolution but excluded from output projection. Filters referencing filter-only dimensions are applied in grain group CTEs only (not in final SELECT).

Test Plan

Updated expected SQL in tests to remove single-argument COALESCE + added TestFilterOnlyDimensions test class for filter-only dimension behavior.

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

@netlify
Copy link
Copy Markdown

netlify bot commented Jan 25, 2026

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit 38ebca3
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/69770a59d61462000864a930

@shangyian shangyian changed the title generate sql without coalesce on single arg Optimize DJ SQL and fix Trino compatibility Jan 26, 2026
@shangyian shangyian marked this pull request as ready for review January 26, 2026 06:57
@shangyian shangyian merged commit 683d9d1 into DataJunction:main Jan 26, 2026
17 checks passed
@shangyian shangyian deleted the djsql-speed branch January 26, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant