feat(spec)!: reject unknown keys on the ETL authoring contracts (#4001 批 12) - #4979
Merged
Merged
Conversation
…批 12) Seven strip sites in `automation/etl.zod.ts` close — `ETLSource` (+ its `.incremental` block), `ETLDestination`, `ETLTransformation`, and `ETLPipeline` (+ its `.retry` and `.notifications` blocks). `automation/` remaining-strip: 53 → 46 (authorable 27 → 20). The file's other three sites — `ETLPipelineRun` + `.stats` + `.error` — are deliberately left OPEN. Every key on them is a fact the engine produces about a run that already happened; nobody authors a run result, so strictness buys no author protection there and would turn a future counter into a parse crash for every existing reader (the #3712 shape). Same disposition, same reason, as `FlowVersionHistorySchema` and all of `execution.zod.ts`. The exemption is recorded on the schema itself and pinned in `etl.test.ts`, not only in the ledger — a note only the ledger carries is a note the next sweep misses. Verified before tightening, and the verification has a stated limit: this file has NO parse site in objectstack / objectui / cloud, so neither half could be settled by pointing at a live call. The seven are authorable because the exported schema and type ARE the authoring door (SYNC_ARCHITECTURE.md and the module's own `@example` both hand-write `const p: ETLPipeline = { … }`) — the same posture the ledger already records for `webhook.zod.ts`. The three are wire on the shapes' semantics plus settled precedent, not on an emit site. Curation is anchored, not invented. With no stored payloads to scan, every alias and guidance entry names a sibling contract in this repo that spells the same intent differently: `timestampField` → `cursorField` (connector `DataSyncConfig`), `onError` → `onFailure` (three in-repo surfaces), `maxRetries` → `maxAttempts` and the retired `retryDelayMs` → `backoffMs` (the converged `shared/RetryPolicySchema`, #4661). The connector's one `strategy` enum splits across two keys here, so it resolves to `writeMode` on a destination and `syncMode` on a pipeline — a single global alias would misdirect one of them with full confidence. `direction` gets a documented absence rather than a rename: an ETL pipeline states direction structurally. The dominant failure on this file is misplacement, not typing — `table`, `endpoint`, `path`, `condition` are real settings one level down, inside the open `config` bag. That pointer lives in `history` (appended to every message) rather than a per-key table, because the open bag's vocabulary is unbounded and enumerating it would be guesswork. One published-artifact consequence, stated rather than glossed: the campaign's "strictness does not move the JSON Schema" claim is per-direction, and `ETLPipelineSchema` is the case where the usual direction does not apply — it cannot convert in output mode (`schedule` is a transform), so build-schemas falls back to input mode, where strip emits nothing and strict emits `additionalProperties: false`. The publication now matches the parse; it is a real narrowing, not a no-op, and it is pinned in both directions. Two out-of-scope findings filed rather than fixed: #4962 (`retry` is a third retry-policy vocabulary #4661's convergence never reached) and #4963 (all nine type aliases export the parsed shape under the bare name, which is why the SYNC_ARCHITECTURE.md pipeline examples do not compile). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ehu85kbvMcrNTUJjwxvLJ9
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ehu85kbvMcrNTUJjwxvLJ9
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
…omation-batch12 # Conflicts: # docs/audits/2026-07-unknown-key-strictness-ledger.md
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 106 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…omation-batch12 # Conflicts: # docs/audits/2026-07-unknown-key-strictness-ledger.md
xuyushun441-sys
marked this pull request as ready for review
August 3, 2026 18:59
xuyushun441-sys
enabled auto-merge
August 3, 2026 18:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #4001 — 批 12,
automation/主体的最后一批。automation/etl.zod.ts的 10 个 strip 点里,7 个收紧、3 个明确留开。本批贡献 −7;叠加期间落地的批 10(#4973)与批 11(#4974)之后,automation/剩余 strip 53 → 26,authorable 27 → 0。收紧的 7 个(授权面)
ETLSourceSchema(含.incremental)、ETLDestinationSchema、ETLTransformationSchema、ETLPipelineSchema(含.retry与.notifications)。留开的 3 个,以及为什么这条豁免写在了三个地方
ETLPipelineRunSchema+.stats+.error保持 tolerant。这些键全部是引擎对已经发生过的一次运行产出的事实:它铸的 id、它到达的 status、它累加的计数器、它捕获的 error。没有人「编写」一份运行结果 —— 手写一份不是用例,是对历史撒谎。所以收紧在这里买不到任何作者保护,却会让「未来引擎多报一个计数器」变成所有既有 reader 的 parse 崩溃(#3712 在HookContextSchema.provenance上的形状)。与FlowVersionHistorySchema和整个execution.zod.ts同一处置、同一理由。豁免记录在 schema 自己的 JSDoc +
etl.test.ts的 pin + ledger 行 三处,而不只在 ledger 里 —— 一条只有 ledger 记得的决定,下一次扫描时和「没人做完」没有区别。批 11 在flow.zod.ts上独立得出了同样的做法(它的FlowVersionHistorySchema豁免现在也落在 schema 旁边 +flow.test.ts),两个并行批次收敛到同一形状,这比任何一边单独主张都更能说明它是对的。verify-before-tightening:结论,以及这次验证的边界
分类与 remeasure 的 7+3 完全一致(
analyzeSites逐点确认)。但验证的方式必须如实说明:etl.zod.ts在 objectstack / objectui / cloud 三仓没有任何 parse 点,ETL 也不可达于ObjectStackDefinitionSchema(实测:stack 根的 JSON Schema 里cursorField/writeMode/ETL一个都不出现)。所以两半都无法靠「指向一次活的调用」来判定。SYNC_ARCHITECTURE.md与本模块自己的@example都在手写const p: ETLPipeline = { … }。这正是 ledger 已经为webhook.zod.ts记录的 spec-only 姿态。curation:没有 payload 可扫时,每条都锚在本仓库存在的兄弟契约上
批 9 的 curation 是从 630 个真实 flow-node payload 扫出来的。这里一个 payload 都没有,所以换了另一种可核验的锚:每条 alias / guidance 都指向仓库里另一个把同一个意图拼成别的词的契约,并在注释里点名。
timestampField→cursorFieldintegration/connector.zod.ts的DataSyncConfig.timestampField(活的 parse 路径),describe 写的是同一件事strategy→writeMode(destination)/syncMode(pipeline)SyncStrategySchema枚举full/incremental/upsert/append_only,四个值恰好劈成两半:写入半边是writeMode,抽取半边是syncModedirection→ 无此键DataSyncConfig.direction。ETL 的方向是结构性表达的(谁是source谁是destination),要反向就交换两端maxRetries→maxAttempts;retryDelayMs→backoffMs;backoffMultiplier/maxRetryDelayMs/jitter为记录在案的缺席shared/retry-policy.zod.ts(#4661 收敛后的唯一声明)onError→onFailureonError(ui/widget.zod.ts、data/hook.zod.ts的声明键表、kernel/plugin-loading.zod.ts)strategy一词在两个面上解析到不同的键,是这批里最值得单独测的一条:一条全局 alias 会以十足的信心把其中一半的作者引到错的键上。测试专门 pin 了这对。ETLTransformationSchema没有 curated table,并在注释里说明原因 —— 沿用HttpConfigSchema(批 9)的先例:凭空发明无人能证伪的条目,正是这个 campaign 曾经一次性发出四条自信错误处方的方式。这个文件的主要失败模式不是拼错,是放错层
source/destination/transformation三者都是「一个小的封闭键集 + 一个开放的config: z.record(…)袋子」。所以未声明键在这里更常是错层而非笔误:table、schema、endpoint、path、format、condition、groupBy都是真实且承重的设置,只是应该在下一层。.strip把它们就地删掉,于是管道 parse 干净、然后拿着一个恰好缺了作者写的那条设置的config去跑。这条指路写在
history里(附加到该面每一条未知键消息上),而不是逐键的guidance表 —— 因为错放的键取自开放袋子的无界词表,逐条枚举就是猜测,而指明目的地不是。ADR-0087:不需要 conversion,而且这个「绿」是先证红过的
三个 example app 的构建产物(
dist/objectstack.json)全量走查:3930 个节点,0 个被新契约拒绝;先注入一个带timestampField的合成 pipeline 作负控,探针确实报红,green 才成立。三个objectstack validate全部通过。探针第一版过匹配了,值得记一笔:
ETLTransformationSchema的必需形状{ type, config }是 flow node 的真子集,且它的type枚举与 flow 节点类型词表在script/map/filter/merge上重叠。于是 showcase 的两个 flow 节点(showcase_task_completed的script、showcase_release_signoff的map)被报成 ETL transformation 的回归。它们不是 —— 没有任何东西把 flow node 路由到 ETL 契约,那两个节点归批 9 的ScriptConfigSchema/MapConfigSchema管。探针改成 provenance-aware 之后如实排除。照第一版数字报告,就会凭空发明一个 blast radius。一处发布产物确实动了,不是 no-op
campaign 的「收紧不改变已发布 JSON Schema」这条断言是按方向成立的:
build-schemas.ts首选io: 'output',而 output 模式下 strip 对象本来就发additionalProperties: false。ETLPipelineSchema恰好是那条断言不适用的情形 —— 它在 output 模式下根本转不出来(schedule是CronExpressionInputSchema,一个 transform,"Transforms cannot be represented in JSON Schema"),于是 build 回退到 input 模式;而 input 模式下 strip 什么都不发、strict 发false。所以发布出来的 pipeline schema 从「未指定」收窄为「封闭」。方向是对的(发布终于和 parse 一致,而不是比 parse 更宽松),但它是一次真实收窄,测试按两个方向都 pin 了。ETLPipelineRun走 output 模式,未移动(实测其required不含带 default 的键,可据此判定走的是哪个方向)。范围外发现(按 Prime Directive #10 立 issue,未在本 PR 修)
ETLPipeline.retryis a third retry-policy vocabulary that #4661 的收敛没有覆盖到 #4962 —ETLPipeline.retry是重试策略的第三份词表,spec 双源清账 C8:RetryPolicy / RetryPolicySchema(./automation ≠ ./system)—— 2 条 #4661 的收敛没扫到它(它是匿名内联 block,没有导出名,不在 spec 同名双源:两个MetadataWatchEvent形状不同、分挂两个子路径入口,其中 kernel 版零消费方(ADR-0049 enforce-or-remove) #4411/✅ spec 双源清账主单:基线 52 → 0(2026-08-03 收官)—— #4446 gate 落地后的偿还 worklist #4535 的雷达上)。三处分歧都是真的:maxAttemptsvsmaxRetries、默认值3vs0(方向恰好与 spec 双源清账 C8:RetryPolicy / RetryPolicySchema(./automation ≠ ./system)—— 2 条 #4661 论证的相反)、少三个键。本 PR 只把这份分歧变吵:那四条 guidance 就是它当前的可见形式。automation/etl.zod.ts的九个类型别名全部导出 parsed 形状,违反X/XParsedhouse convention(SYNC_ARCHITECTURE.md 的示例因此不可编译) #4963 — 九个类型别名全部把 parsed 形状占用了裸名,违反X/XParsedhouse convention。后果具体可查:SYNC_ARCHITECTURE.md的两段 pipeline 示例不可编译,而这个文件唯一的授权门恰恰就是「用ETLPipeline标注手写字面量」。两条都给了双轴分析和建议,都需要维护者裁决,因此没有塞进这个 strictness 批次里顺手做掉。
Ledger,以及两轮串行同步
etl.zod.ts的 triage 行改为mixed并写明 7/3 与分类边界;remaining-strip 行10 | 10 | mixed→3 | 10 | wire。冲突按预期发生了两次 —— 开 PR 后批 10 落地,随后批 11 落地 —— 每次都是行干净合并、只有表头与其下的小计冲突。两次都按同一条规则解决:保留各方的行编辑,由存活的行重算表头与小计,谁的数都不采信,再让
check:strictness-ledger的算术裁决。最终存活行:
execution13 +etl3(本批) +flow1(批 11) +bpmn-interop5 +node-executor4 = 26 strip of 75,authorable 0。散文按双方内容合成:保留批 11 的 wave 表(补上批 12 一行)与它记录的「不冲突的小计反而在每个分支上都是错的」这条教训 —— 并把计数从三次更新为四次,同时点明批 12 一家就撞了两次(对批 10 一次、对批 11 一次),所以规律不是「每批一次」,而是每一对在飞行中重叠的批次一次。
并修正了本 PR 自己先前写错的一句:原文声称 etl 是
automation/里第一个「缩小但不消失」的行 —— 批 11 的flow.zod.ts(7 → 1)先做到了,且先落地。已改为第二个,并说明两批是并行独立收敛到同一形状的。验证(批 11 合并之后重跑,scoped + flock)
新增 25 个测试。三次 sabotage 证明仪器会红:把
ETLDestinationSchema改回 strip → 4 红;把ETLPipelineRunSchema收紧 → 2 红(豁免 pin 生效);把 destination 的strategyalias 改成syncMode→ 1 红。恢复后全绿。os-regen 四步在三次 merge 后各走了一遍,三次全量重生成均产出零 diff;每次都断言各方条目共存 —— 本批 35 条
automation/ETL*+ 19 个 ETL 导出;批 11 的四张参考页(flow/flow-function/time-relative-trigger/webhook.mdx)与 webhook 信封 19 条 +TimeRelativeTrigger6 条 +FlowNode11 条;批 10 的StateNode8 条与StateMachine/ControlFlow导出;批 9 的NotifyConfigSchema/ScriptConfigSchema。⛔ 仍为 Draft,由 coordinator ready + auto-merge。