fix(driver-sql,analytics): aggregate()/distinct() 不再把 SQLite 的裸 epoch 漏出去 (#3797) - #3804
Merged
Merged
Conversation
…'s raw epoch storage (#3797) Both returned `await builder` directly, without the `formatOutput` pass every `find()` row gets. On SQLite — the one dialect where a `Field.datetime` is stored as INTEGER epoch milliseconds rather than a native timestamp — that raw storage form went straight to the caller, while the same column read through `find()` came back as canonical ISO-Z. Same root cause as #3773, different exit. `Field.date` was never affected: it is ISO TEXT on every dialect, so its storage form already equals its presentation. The visible surfaces were a `_max`/`_min` measure over a datetime (a "last closed" KPI tile rendered 1768035600000) and a `groupBy` on a raw datetime dimension, which also disagreed with the in-memory `applyInMemoryAggregation` fallback — that one consumes already-formatted `find()` rows, so the same dataset changed key type depending on which path served it. Which columns hold an instant is recorded while the statement is built, because that is the only point where a column name and its meaning are both known: a `min()` lands under its alias and never under the field name, while a date-BUCKETED column lands under the field name but holds a label ('2026-01') rather than an instant. Matching on names afterwards gets both backwards, and there is a test for each direction. `distinct()` additionally re-deduplicates after presenting: SQL DISTINCT compares STORED values and one SQLite datetime column holds both forms, so two rows recording the same instant survived as two and then presented identically. It has no in-repo callers today; fixed for consistency, not urgency. cross-object-rebucket is fixed alongside it, because presenting min/max correctly is what exposed it. `recombine()` coerced every operand with `Number()`, which silently depended on receiving an epoch: handed the ISO string the driver now returns it produced NaN, and on Postgres/MySQL (where knex returns a Date) it had always flattened the value back to an epoch integer one layer above the driver. min/max now order by the instant and return the winning value in the shape it arrived in; sum/count stay numeric. Verified red first — the temporal cases produce NaN against the old implementation. Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 2 package(s): 13 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3797。#3773 的直接续作 —— 同一个根因(绕过
formatOutput的读路径,在唯一「存储 ≠ 呈现」的方言上漏原始形态),换了个出口。症状
aggregate()和distinct()都直接return await builder,不过formatOutput:find()"2026-01-10T09:00:00.000Z"distinct('closed_at')[1768035600000]["2026-01-10T09:00:00.000Z"]aggregate()max(closed_at)1768035600000"2026-01-10T09:00:00.000Z"aggregate()groupBy: ['closed_at']1768035600000"2026-01-10T09:00:00.000Z"Field.date从来不受影响 —— 它在每个方言上都是 ISO TEXT,存储形态本来就等于呈现形态。可达面:
_max/_min度量(inferMeasure认这两个后缀,所以「最近成交时间」KPI 磁贴显示1768035600000),以及对裸 datetime 维度的groupBy—— 后者同时是一条 parity 漂移:内存兜底applyInMemoryAggregation吃的是已formatOutput过的find()行,同一个数据集换条路走桶键类型都不一样。这正是 #3773 为 date-bucket 那一半刚钉下的契约。修法上的陷阱(两个方向各有一条测试)
不能事后按列名匹配声明字段,两头都会错:
min()的结果落在 alias 下(latest),永远不叫字段名 —— 按名字找根本够不着。Field.datetime的 dateGranularity 分桶恒为 NULL —— 趋势图塌成一根柱子 #3773 之后,带dateGranularity的分桶列恰恰叫字段名,但它的值是标签('2026-01')不是实例 —— 按名字规范化会把标签喂进 datetime 呈现器。所以「哪些列装的是实例」在建语句时就记下来,那是列名和语义同时已知的唯一时刻。
distinct()另外做了呈现后去重:SQLDISTINCT比的是存储值,而一个 SQLite datetime 列同时装 INTEGER 和 TEXT 两种形态,同一个实例的两行能双双活过DISTINCT然后呈现成同一个值。它目前全仓零调用方,修它是为了不在驱动里留第二套约定,不是紧急度。顺带修掉一个我自己会造出来的回归
cross-object-rebucket的recombine()对每个操作数Number()强转,这是在暗中依赖收到的是 epoch:Number('2026-01-10T…')=NaN;Date)它一直把值拍回 epoch 整数 —— 等于在驱动上面一层又漏了一次。改成:
min/max按实例排序、返回进来时那个形状的获胜值;sum/count保持数值。先验红:把新逻辑关掉重跑,两条时态用例如实吐出NaN。这条不是顺手扩范围 —— 是「把 min/max 呈现对」这个改动直接制造出来的回归,归我修。
测试
新增
sql-driver-aggregate-temporal-output.test.ts(15 条),先红后绿:未修时 8 红 6 绿,而绿的正好是本来就该对的那几条(date 列、非时态列、分桶标签、数值聚合),说明测试有鉴别力而不是一律断言。覆盖distinct/min+max/ 裸groupBy/ 结构化无 granularity 的groupBy/ 混合groupBy/ 与find()的一致性 / 与内存兜底键的一致性 / 混合存储去重,外加两条反向断言(分桶标签不被规范化、数值聚合不被规范化)。cross-object-rebucket.test.ts加两条(ISO 字符串、Date对象)。driver-sql 401 ✅ / service-analytics 269 ✅ / driver-sqlite-wasm 126 ✅;
turbo build --force(绕开缓存跑真 DTS,因为OS_SKIP_DTS不进缓存键、本会话早前用它建过 driver-sql)通过;eslint 干净。