Skip to content

Rust duckdb extension#5

Merged
nicosuave merged 17 commits intomainfrom
rust-duckdb-extension
Nov 29, 2025
Merged

Rust duckdb extension#5
nicosuave merged 17 commits intomainfrom
rust-duckdb-extension

Conversation

@nicosuave
Copy link
Copy Markdown
Member

No description provided.

sidemantic-rs: Rust implementation using sqlparser-rs
- Core types: Model, Dimension, Metric, Relationship, Segment
- SemanticGraph with BFS join path discovery
- SQL generation and query rewriting
- YAML config loading (native + Cube.js formats)
- C FFI bindings for DuckDB integration
- 29 passing tests

sidemantic-duckdb: DuckDB extension wrapping Rust library
- Parser extension: SEMANTIC keyword triggers query rewriting
- Table functions: sidemantic_load(), sidemantic_models()
- Scalar function: sidemantic_rewrite_sql()
- Links static Rust library via C FFI
…ension

Rust library enhancements:
- Add cumulative metrics (running totals, period-to-date MTD/YTD)
- Add time comparison metrics (YoY, MoM, WoW growth)
- Add fill_nulls_with and format options to metrics
- Improve error messages with available options (model/dimension/metric/segment not found)
- Add helper functions for creating typed errors

DuckDB extension:
- Add sidemantic_load_file(path) table function to load YAML from files or directories
- Auto-generate LEFT JOINs when querying across related models
- Detect fan-out risk when joining one-to-many and add SQL warning
- Support custom join conditions via Relationship.sql field
- Add {from}/{to} placeholders for custom join SQL
- Wire custom conditions through SemanticGraph adjacency list
- Add tests for fan-out detection, custom joins, and cross-model queries
…README

- Add Cross-Model Queries section with auto-JOIN example
- Document sidemantic_load_file() function for file/directory loading
- Add relationship types table with fan-out risk indicators
- Document custom join conditions with {from}/{to} placeholders
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sidemantic-rs/src/sql/rewriter.rs Outdated
@nicosuave nicosuave force-pushed the rust-duckdb-extension branch from 0a8040c to 6c42a41 Compare November 26, 2025 20:01
- Add separate Rust CI job for sidemantic-rs (fmt, clippy, test)
- Rename Python job for clarity
- Fix all clippy warnings (uninlined format args, type complexity, etc.)
- Add type alias for adjacency edges to reduce complexity
- Add module-level allow for FFI raw pointer derefs
- Delete unused NEXT_README.md template boilerplate
- Add TableCalculation struct with types: running_total, percent_of_total,
  percent_of_previous, rank, dense_rank, row_number, moving_average,
  difference, lead, lag, formula
- Generate SQL window functions instead of post-processing (more efficient)
- Support partition_by, order_by, window_size, offset parameters
- Add to SemanticQuery with with_table_calculations() builder
- Wire into SqlGenerator to include in SELECT clause
…tion parser

Features:
- Model inheritance: extends field, resolve_model_inheritance()
- Symmetric aggregates: auto-applied in generator for fan-out prevention
- Relative dates: expand 'last 7 days', 'this month', etc. in filters
- SQL definition parser using nom for MODEL/DIMENSION/METRIC/SEGMENT syntax

Wiring:
- Symmetric aggregates used automatically when fan-out detected (no more warning)
- Inheritance resolved during YAML loading
- Relative date expressions expanded in filter processing
- SQL parser available via parse_sql_model() and parse_sql_definitions()
New features:
- Simple SQL-like syntax: METRIC revenue AS SUM(amount)
- SEMANTIC MODEL command to switch active model context
- model.metric prefix syntax for explicit model targeting
- Fix COUNT(*) metric generation (was COUNT(metric_name))

Parser improvements:
- Added sqlparser-rs integration for expression parsing
- Support multiple syntax variants in alt() combinator
- Handle METRIC/DIMENSION model.name (props) syntax
@nicosuave nicosuave force-pushed the rust-duckdb-extension branch from 82811f9 to 4b7158e Compare November 27, 2025 01:31
@nicosuave nicosuave force-pushed the rust-duckdb-extension branch from 4b7158e to cf1a905 Compare November 27, 2025 01:38
The workflows in sidemantic-duckdb/.github/ never ran since GitHub
only checks the root .github/ directory. Added a new duckdb-extension
job to the main CI that builds Rust first, then builds and tests the
DuckDB extension.
New syntax requires CREATE keyword:
- SEMANTIC CREATE METRIC revenue AS SUM(amount)
- SEMANTIC CREATE OR REPLACE METRIC revenue AS SUM(amount)
- SEMANTIC CREATE DIMENSION status AS status

The OR REPLACE flag is parsed but not yet implemented in Rust.
- Add SEMANTIC CREATE OR REPLACE METRIC/DIMENSION/SEGMENT syntax
- Add model.name dot notation for explicit model targeting in AS syntax
- Add Expression aggregation type for complex expressions like SUM(amount) * 2
- Update sidemantic_add_definition FFI to accept is_replace parameter
- Implement replace logic using retain() to remove existing items by name
@nicosuave nicosuave merged commit 9cc55df into main Nov 29, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant