Skip to content

Fix aggregation parsing for complex expressions#31

Merged
nicosuave merged 1 commit intomainfrom
rill-kitchen-sink-fixes
Jan 3, 2026
Merged

Fix aggregation parsing for complex expressions#31
nicosuave merged 1 commit intomainfrom
rill-kitchen-sink-fixes

Conversation

@nicosuave
Copy link
Copy Markdown
Member

Summary

  • Replace greedy regex with sqlglot in Metric class to properly handle expressions like SUM(x) / SUM(y) without mangling them
  • Fix COUNT DISTINCT detection using isinstance(parsed.this, exp.Distinct)
  • Add expression metric support in SQL generator for metrics with inline aggregations
  • Fix dependency analyzer to skip resolution for expression metrics with inline aggregations
  • Fix cumulative metrics to properly resolve references to other measures and generate valid aliases
  • Add kitchen sink tests using patterns from rill-examples to catch edge cases

Bugs Fixed

  1. Greedy regex: SUM(deletions) / SUM(changes) was being parsed as agg=sum, sql=deletions) / SUM(changes
  2. COUNT DISTINCT: COUNT(DISTINCT user_id) was parsed as agg=count, sql=DISTINCT user_id instead of agg=count_distinct, sql=user_id
  3. Expression metrics: Metrics with full SQL expressions (no agg, no type) couldn't be queried
  4. Window functions over measures: Cumulative metrics referencing other measures failed to resolve

Test Results

  • 42/42 Rill adapter tests pass
  • 25/25 new kitchen sink tests pass

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 11a079a1d9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread sidemantic/core/metric.py Outdated
Comment on lines +107 to +111
if isinstance(parsed.this, exp.Distinct):
agg_func = "count_distinct"
inner_expr = distinct_match.group(1).strip()

data["agg"] = agg_func
data["sql"] = inner_expr
# Extract the actual expression from inside Distinct
if parsed.this.expressions:
inner_expr = parsed.this.expressions[0].sql(dialect="duckdb")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve all DISTINCT expressions in COUNT

When COUNT(DISTINCT ...) has multiple expressions (e.g., COUNT(DISTINCT a, b)), sqlglot exposes them in Distinct.expressions. This code now only takes expressions[0], which drops the remaining expressions and changes the aggregation semantics (distinct on a instead of (a,b)), so counts will be too high for multi-column DISTINCTs. The previous regex path preserved the full a, b list, so this is a regression for any multi-column distinct usage.

Useful? React with 👍 / 👎.

@nicosuave nicosuave force-pushed the rill-kitchen-sink-fixes branch 2 times, most recently from 76070d1 to 1aee662 Compare January 3, 2026 21:37
- Replace greedy regex with sqlglot in Metric class to properly handle
  expressions like SUM(x) / SUM(y) without mangling them
- Fix COUNT DISTINCT detection using isinstance(parsed.this, exp.Distinct)
- Add expression metric support in SQL generator for metrics with inline
  aggregations (agg=None, type=None, sql=<expression>)
- Fix dependency analyzer to skip resolution for expression metrics with
  inline aggregations
- Fix cumulative metrics to properly resolve references to other measures
  and generate valid aliases

Add kitchen sink tests using patterns from rill-examples to catch edge cases.
@nicosuave nicosuave force-pushed the rill-kitchen-sink-fixes branch from 1aee662 to 3c2ecef Compare January 3, 2026 21:48
@nicosuave nicosuave merged commit b87b32c into main Jan 3, 2026
9 of 10 checks passed
@nicosuave nicosuave deleted the rill-kitchen-sink-fixes branch January 3, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant