feat(verify): checkDateBucketParity —— 钉住「下推 SQL 分桶 ↔ 内存分桶」那条没人守的接缝 (#3773 收尾) - #3813
Merged
Conversation
…n and in-memory date bucketing A driver that advertises `supports.queryDateGranularity[g]` is telling `engine.aggregate` it may push that granularity down as SQL instead of fetching rows and bucketing them in JS. The two are then not two features but one feature with two implementations, and the engine picks between them per query — a granularity the driver advertises goes down as SQL, one it does not goes to `applyInMemoryAggregation`, and a non-UTC timezone forces the in-memory path regardless. A dashboard can cross that seam mid-drill-down. Nothing checked that they agree. That is how #3773 shipped: every gate was green while a trend chart collapsed into one bar, because the driver's own bucket suites build fixtures with `knex.schema.createTable` + `t.string(...)` — ISO TEXT, the half `strftime` parses natively — and the engine never second-guesses a granularity a driver claims to support. `checkDateBucketParity(driver)` compares the driver's pushed-down result against the REAL `applyInMemoryAggregation` over the driver's own `find()` rows, for every granularity it advertises, across both temporal storage forms under one object (`Field.datetime` and `Field.date` naming the same calendar days) so a storage-form leak surfaces as the two columns bucketing differently. A granularity the driver does not advertise is skipped, never faulted. Follows `checkReadCoercion`: human-readable problems (empty = conformant), no test-runner dependency, driver taken structurally — so cloud's driver-turso, remote SQLite with exactly the epoch storage that broke here, can run the identical contract against itself. Verified against the real regression, not just fakes: reverting the #3773 fix turns the gate red on BOTH drivers with a diagnostic naming the collapsed bucket (`null: sql=7 in-memory=—`), and the cross-column pass fires alongside it while the TEXT-stored date column correctly stays quiet. Four negative controls pin what the checker can detect, including a driver that advertises a granularity it then throws on, and one that advertises nothing (must stay silent). The three hand copies of `bucketDateValue` now state what their `⚠️ Keep in sync` comments cannot enforce — a copy that stops tracking its original leaves the copy and the SQL agreeing with each other while both are wrong — and point at the executable check. The same pointer is on `bucketDateValue` itself, where an edit would start the drift. Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 5 package(s): 21 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#3773 / #3797 的收尾:把那道没人守的接缝钉住,不修任何现有 bug。
先纠正我自己的提法
我原先说要加「文本一致性门」—— 假设三份
bucketDateValue手抄件是逐字复制。查了,不是。 副本彼此逐字相同,但和 objectql 原版根本不同:原版带timezone参数、走calendarPartsInTzOrUtc,副本是 UTC-only 内联版。文本门对着源头开箱就假阳性。而真正的风险方向也不是"副本乱改":副本改坏了,驱动测试会红(SQL 对不上改坏的副本)。危险的是改了 objectql 原版、副本没跟 —— 此时副本和 SQL 彼此自洽、双双是旧的,全绿。注释治不了这个,只有行为门能。
门是什么
checkDateBucketParity(driver)(@objectstack/verify):对驱动声明支持的每个粒度,拿它下推的 SQL 结果去比真的applyInMemoryAggregation跑它自己find()行的结果。契约的措辞很重要:驱动
supports.queryDateGranularity[g]是在告诉engine.aggregate「这个粒度可以下推」。于是它俩就不是两个特性,是一个特性的两份实现,引擎按查询逐次挑一份 —— 声明了的走 SQL,没声明的走内存,非 UTC 时区一律走内存。仪表盘的一次 drill-down 就能跨过这条缝。Field.datetime和Field.date指同一批日历日,于是"存储形态泄漏"会表现为两列分桶不一致,即使各自和自己的内存参考都自洽。形状照抄
checkReadCoercion:返回人类可读的 problems(空 = 合规)、不依赖 test runner、结构化接受任意驱动 —— 所以 cloud 的driver-turso能拿去跑同一份契约。这条最重要:Turso 是远程 SQLite,正是这次出事的那种 epoch 存储。验红:拿真回归验,不是只拿假驱动
把 #3773 的修复退回去重跑,门在两个驱动上都红,诊断直接点名塌掉的桶:
跨列那道也同时开火(8 条 = 4 粒度 × 2 驱动),而 per-column 那道只点名
'at',从没误伤 TEXT 存储的'on'。恢复后全绿,git diff对驱动源码为空。另配 4 条反向对照钉住 checker 的能力边界:全塌成 null(#3773 原形)、声明了却抛错、一列对一列塌(存储形态泄漏)、什么都不声明必须保持沉默。
注释改了两处,分工写清楚
三份手抄件的
⚠️ Keep in sync现在明说它自己检测不了漂移(副本和 SQL 会彼此自洽地一起错),并指向可执行的门。同样的指针也加在bucketDateValue本体上 —— 那才是漂移的起点,改的人在那儿。一处刻意不覆盖
fixture 里没有 null 实例。NULL 分桶键是既有分歧(SQL 给 SQL NULL,内存给字面量
'(null)'),与分桶正确性正交、在 TEXT 列上同样成立。放进来会让门为一个它不负责的理由而红。这一点写在 fixture 注释里,好让"没有 null"读起来像决定而不是疏忽。测试
新门 6 条 ✅(2 真驱动 + 4 反向对照);driver-sql 401 ✅ / objectql 1128 ✅ / sqlite-wasm 126 ✅ / verify 7 ✅;
turbo build --force(绕缓存跑真 DTS)通过;eslint 干净。dogfood 全量跑有 2 条既有红:
attachments-permission-matrix的两条AUTH_REQUIRED。已用git stash隔离到纯净 origin/main 上同样红,与本 PR 无关(本分支基点 9e4bad5 正是改那个附件 harness 的提交)。