feat: implement aggregation methods with support for grouping and med…#577
Conversation
…ian calculations in Clickhouse, MongoDB, and MySQL connectors
There was a problem hiding this comment.
Pull request overview
Adds getAggregateWithOriginalTypes support to additional data connectors so resource().aggregate() can run server-side aggregations (sum/count/avg/min/max/median) with optional grouping by field or date truncation.
Changes:
- Implemented aggregation query building + grouping for MySQL, MongoDB, and ClickHouse connectors.
- Added connector-specific median implementations (ClickHouse quantile-based; MySQL/Mongo compute median in application code).
- Updated ClickHouse connector typing for
getDataWithOriginalTypesreturn type.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
adminforth/dataConnectors/mysql.ts |
Adds aggregate query generation with grouping and a JS-computed median fed by GROUP_CONCAT. |
adminforth/dataConnectors/mongo.ts |
Adds MongoDB aggregation pipeline for group + aggregates, including JS-side median from $pushed values. |
adminforth/dataConnectors/clickhouse.ts |
Adds ClickHouse aggregate query generation with grouping and median via quantile(0.5). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ian calculations in Clickhouse, MongoDB, and MySQL connectors
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… return types and improve date truncation handling
…on variable for group_concat_max_len
…nctions and improve performance
…endpoints for car statistics
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| case 'avg': groupStage[alias] = { $avg: { $toDouble: `$${rule.field}` } }; break; | ||
| case 'min': groupStage[alias] = { $min: { $toDouble: `$${rule.field}` } }; break; | ||
| case 'max': groupStage[alias] = { $max: { $toDouble: `$${rule.field}` } }; break; | ||
| case 'median': groupStage[alias] = { $push: { $toDouble: `$${rule.field}` } }; break; |
There was a problem hiding this comment.
Mongo median implementation uses $group + $push to accumulate all values for the field into an array per group, then sorts to compute the median. This can easily exceed MongoDB's 16MB document limit for a group and/or the aggregation memory limits on large datasets, causing the query to fail or become unstable. Please compute the median fully inside MongoDB without materializing the entire value list (e.g., using a server-side median/percentile operator when available, or a window-function based approach), or otherwise implement a bounded/approximate median strategy.
| case 'median': groupStage[alias] = { $push: { $toDouble: `$${rule.field}` } }; break; | |
| case 'median': groupStage[alias] = { $median: { input: { $toDouble: `$${rule.field}` }, method: 'approximate' } }; break; |
…rove groupBy handling
…ian calculations in Clickhouse, MongoDB, and MySQL connectors