feat(metrics): add MEDIAN/STDDEV_SAMP/VAR_SAMP as system-wide aggregates - #42895
Open
rusackas wants to merge 4 commits into
Open
feat(metrics): add MEDIAN/STDDEV_SAMP/VAR_SAMP as system-wide aggregates#42895rusackas wants to merge 4 commits into
rusackas wants to merge 4 commits into
Conversation
github-actions
Bot
added
preset-io
doc
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
rusackas
changed the title
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #42895 +/- ##
=======================================
Coverage 66.40% 66.41%
=======================================
Files 2857 2857
Lines 161303 161357 +54
Branches 37133 37137 +4
=======================================
+ Hits 107120 107160 +40
- Misses 52158 52171 +13
- Partials 2025 2026 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
rusackas
marked this pull request as ready for review
August 8, 2026 04:21
dosubot
Bot
added
change:backend
…regates [DRAFT/SIP] Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tes [DRAFT/SIP] Implements the proposal in docs/sip/median-stddev-variance-aggregates.md: MEDIAN, STDDEV_SAMP, and VAR_SAMP are now selectable anywhere a metric aggregate is chosen, not just Pivot Table. Support is opt-in per engine via a new BaseEngineSpec.get_extended_aggregation_func hook (mirroring supports_grouping_sets/_time_grain_expressions), verified against live Postgres/MySQL/DuckDB instances before being enabled; unsupported engines raise a clear error instead of emitting unverified SQL. No pivot-table- specific code needed: these aggregates fall outside ADDITIVE_AGGREGATES automatically, so totals/subtotals already route through the correct DB-rollup path SIP-216 (#41184) introduced. Also fixes a live bug this surfaced: the MCP chart-creation service advertised STDDEV/VAR as valid aggregates that never matched any real Superset aggregate name, so MCP-built charts using them always errored at query time. Now normalized to STDDEV_SAMP/VAR_SAMP. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… decided Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
rusackas
force-pushed
the
sip/median-stddev-variance-aggregates
branch
from
August 8, 2026 05:20
8305f5e to
f02ef22
Compare
![codeant-ai-for-open-source[bot]](https://avatars.githubusercontent.com/u/151810221?s=60&v=4){kind=small}
Contributor
Code Review Agent Run #ecf60aActionable Suggestions - 0Additional Suggestions - 1
Filtered by Review RulesBito filtered these suggestions based on rules created automatically for your feedback. Manage rules.
Review Details
Bito Usage GuideCommands Type the following command in the pull request comment and save the comment.
Refer to the documentation for additional commands. Configuration This repository uses Documentation & Help AI Code Review powered by |
MEDIAN/STDDEV_SAMP/VAR_SAMP were only verified against live Postgres,
MySQL, DuckDB, and Redshift instances, but `_extended_aggregations`
lived on `PostgresBaseEngineSpec`/`PostgresEngineSpec`/`MySQLEngineSpec`,
so every other subclass (Vertica, Netezza, HANA, Snowflake, CockroachDB,
Greenplum, RisingWave, YugabyteDB, Hologres, Doris, StarRocks, OceanBase)
silently inherited unverified aggregate SQL instead of the intended
"not supported on this database" error. Each now explicitly overrides
`_extended_aggregations` to `{}` until someone verifies it live.
Also fixes two smaller gaps the same review surfaced: MEDIAN was missing
from the MCP chart format validator's numeric-format branch, and
`get_metric_type_from_column`/`METRIC_MAP_TYPE` didn't recognize
STDDEV_SAMP/VAR_SAMP, so an all-null result column for either aggregate
fell back to STRING instead of NUMERIC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Code Review Agent Run #2c7e2aActionable Suggestions - 0Additional Suggestions - 1
Review Details
Bito Usage GuideCommands Type the following command in the pull request comment and save the comment.
Refer to the documentation for additional commands. Configuration This repository uses Documentation & Help AI Code Review powered by |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SUMMARY
This now has a working implementation alongside
docs/sip/median-stddev-variance-aggregates.md, so reviewers (and PMC members this gets shared with) have a concrete design to react to rather than just a description of one. Still opened as a draft: the point is to pressure-test the framing and scope before this goes up for a formal SIP vote, not to pre-empt that discussion.Prompted by a report that Pivot Table lost several aggregation options somewhere along the way. Short version: most of what looked missing (Count Unique Values, Count/Average/Max/Min, "as fraction of" display) already has an equivalent today, see the doc for specifics.
MEDIAN,STDDEV_SAMP(Sample Standard Deviation), andVAR_SAMP(Sample Variance) are genuinely gone, system-wide, not just Pivot Table, since #41184 (SIP-216) deliberately removed the old pivot-only "Aggregation function" control as the mechanism behind our long-standing wrong-totals bug. Not purely hypothetical either: the MCP chart-creation service already advertisedSTDDEV/VARas valid aggregates that never matched any real Superset aggregate name, so MCP-built charts using them always errored at query time — fixed here too, normalized to the real names.What's implemented:
MEDIAN/STDDEV_SAMP/VAR_SAMPadded to the system-wideAggregatetype and the metric-picker dropdown (every chart type, not a Pivot-Table-specific control).BaseEngineSpec.get_extended_aggregation_funcextensibility point (mirrorssupports_grouping_sets/_time_grain_expressions, both from feat(table/pivot-table): correct non-additive totals/subtotals via DB rollup [SIP-216] #41184) — engines opt in only once verified, unsupported engines get a clear "not supported on this database" error instead of unverified SQL.VARIANCE()is population variance, not sample).ADDITIVE_AGGREGATESautomatically, so totals/subtotals already route through the correct DB-rollup path SIP-216 introduced — same mechanism that already makesAVG/COUNT_DISTINCTcorrect today.STDDEV/VARshorthand normalized toSTDDEV_SAMP/VAR_SAMP.Explicitly out of scope, per the doc:
First/Last(no unambiguous SQL meaning as a plain aggregate without an explicit ordering) andList Unique Values(dialect + unbounded-payload questions) — flagged as candidate follow-up SIPs.TESTING INSTRUCTIONS
Backend:
pytest tests/unit_tests/db_engine_specs/test_base.py tests/unit_tests/db_engine_specs/test_postgres.py tests/unit_tests/db_engine_specs/test_mysql.py tests/unit_tests/db_engine_specs/test_duckdb.py tests/unit_tests/db_engine_specs/test_redshift.py tests/unit_tests/models/helpers_test.py tests/unit_tests/mcp_service/Frontend:
npx jest plugins/plugin-chart-pivot-table/test/plugin/utilities.test.ts(confirms the new aggregates are automatically treated as non-additive, with no pivot-table code changes).Manually: on a Postgres-backed dataset, create a chart with a
MEDIAN/STDDEV_SAMP/VAR_SAMPmetric; on a SQLite-backed one, confirm the same aggregate now returns a clear "not supported on this database" error rather than a generic one.ADDITIONAL INFORMATION