Skip to content

ISS-268999: add aliasing to solve Ambiguous column reference#219

Merged
shriram-devrev merged 1 commit intomainfrom
ISS-268999
Mar 16, 2026
Merged

ISS-268999: add aliasing to solve Ambiguous column reference#219
shriram-devrev merged 1 commit intomainfrom
ISS-268999

Conversation

@shriram-devrev
Copy link
Copy Markdown
Contributor

@shriram-devrev shriram-devrev commented Mar 10, 2026

ISS-268999

Add SQL column qualification for TableSchema measures and dimensions

Summary

  • Adds a new utility to qualify unqualified column references in TableSchema measure/dimension SQL expressions by prepending the table name (e.g. customer_idorders.customer_id)
  • Uses DuckDB's json_serialize_sql / json_deserialize_sql for proper AST-based parsing and serialization, with batched processing per table for performance
  • Exposes ergonomic qualifyTableSchemas() wrappers from both meerkat-node and meerkat-browser

Architecture

The implementation is split into four layers:

┌─────────────────────────────────────────────────────────────┐
│  meerkat-node / meerkat-browser                             │
│  qualifyTableSchemas(tableSchemas)                          │
│  → creates DuckDB adapter, wires everything together        │
├─────────────────────────────────────────────────────────────┤
│  meerkat-core: qualifyTableSchemasSql                       │
│  → iterates tables, collects qualifiable members,           │
│    skips {MEERKAT}. placeholders, batches per table          │
├─────────────────────────────────────────────────────────────┤
│  meerkat-core: qualifySqlExpressionColumnsBatch             │
│  → parses SQL → walks AST to qualify COLUMN_REF nodes       │
│    → serializes back only if changes were made              │
├─────────────────────────────────────────────────────────────┤
│  meerkat-core: createDuckDBSqlExpressionAdapter             │
│  → DuckDB-backed parse/serialize via json_serialize_sql     │
│    and json_deserialize_sql, self-contained with batching   │
└─────────────────────────────────────────────────────────────┘

Data flow (per table)

  1. CollectqualifyTableSchemasSql shallow-copies the input TableSchema, collects all measures and dimensions whose SQL doesn't contain {MEERKAT}. placeholders
  2. Parse — All SQL expressions for a table are combined into a single SELECT expr1, expr2, ... and sent to DuckDB's json_serialize_sql in one call. The returned AST has query_location metadata stripped at parse time. Individual ParsedExpression[] are extracted from the select_list
  3. QualifyqualifySqlExpressionColumnsBatch walks each parsed expression AST in-memory. Single-part COLUMN_REF nodes (e.g. ['customer_id']) are rewritten to ['tableName', 'customer_id']. Already-qualified refs (['orders', 'id']), lambda-bound variables, and whitespace-containing identifiers are skipped
  4. Serialize — If any expressions changed, the modified ParsedExpression[] are wrapped in a synthetic SELECT statement with unique batch aliases (__meerkat_batch_expr_0__, etc.), passed to DuckDB's json_deserialize_sql in one call, and the returned SQL string is split back into individual expressions using the alias markers
  5. Apply — Qualified SQL strings are written back onto the shallow-copied member objects

Key design decisions

  • Immutability: Input TableSchema objects are never mutated; a shallow copy is returned
  • Batching: One json_serialize_sql + one json_deserialize_sql call per table (not per expression)
  • Skip serialization when nothing changed: If no COLUMN_REF was qualified, the serialize step is skipped entirely
  • Self-contained adapter: The DuckDB adapter builds json_serialize_sql / json_deserialize_sql queries directly without modifying the shared ast-serializer / ast-deserializer modules
  • query_location stripping: DuckDB adds query_location metadata during parsing that breaks json_deserialize_sql. This is stripped once at parse time via an iterative in-place walk
  • Placeholder safety: Expressions containing {MEERKAT}. are skipped since they aren't valid SQL until placeholder replacement

Consumer usage

Node

import { qualifyTableSchemas } from '@devrev/meerkat-node';

const qualified = await qualifyTableSchemas(tableSchemas);

Browser

import { qualifyTableSchemas } from '@devrev/meerkat-browser';

const qualified = await qualifyTableSchemas({ connection, tableSchemas });

Comment thread meerkat-browser/src/ensure-table-schema-alias/ensure-table-schema-alias.ts Outdated
Comment thread meerkat-core/src/utils/qualify-table-schemas-sql.ts Outdated
Comment thread meerkat-core/src/utils/create-duckdb-sql-expression-adapter.ts Outdated
Comment thread meerkat-core/src/utils/create-duckdb-sql-expression-adapter.ts Outdated
Comment thread meerkat-core/src/utils/create-duckdb-sql-expression-adapter.ts Outdated
Comment thread meerkat-core/src/utils/ensure-sql-expression-column-alias.ts
@shriram-devrev shriram-devrev force-pushed the ISS-268999 branch 3 times, most recently from 0831c3a to fe5886e Compare March 13, 2026 12:45
Comment thread meerkat-core/src/ast-validator/tests/test-data.ts
Comment thread meerkat-core/src/ast-validator/tests/utils.spec.ts
Comment thread meerkat-node/src/__tests__/ensure-sql-expression-column-alias.spec.ts Outdated
@shriram-devrev shriram-devrev enabled auto-merge (squash) March 16, 2026 06:12
@shriram-devrev shriram-devrev merged commit 447e2dd into main Mar 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants