Skip to content

test(automation): reconcile the control-flow designer forms against their Zod (#4045) - #4078

Merged
os-zhuang merged 1 commit into
mainfrom
claude/console-screen-flow-submit-jfpjy4
Jul 30, 2026
Merged

test(automation): reconcile the control-flow designer forms against their Zod (#4045)#4078
os-zhuang merged 1 commit into
mainfrom
claude/console-screen-flow-submit-jfpjy4

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

#4045 step C, region half — after #4050 (step 1), #4059 (3a) and #4064 (step A). Tests only, no production code.

The gap

loop / parallel / try_catch each carry two descriptions of the same config:

artifact job
descriptor configSchema (hand-written literal) drives the Studio form what an author can set
control-flow.zod.ts validates the value what the runtime accepts

Nothing compared them. #4064 pinned the shape of the first; this pins the relationship, so neither side can gain or lose a key without the other noticing.

Both directions matter:

Why the two sources are not merged — measured, not assumed

I ran the generation on main 53fbf49 (options as in control-flow.test.ts):

node published form generated from Zod
loop 597 chars, depth 5, 25 keys 5,537, depth 14, 201 keys 9.3×
parallel 294 chars, depth 6, 16 keys 5,112, depth 15, 189 keys 17.4×
try_catch 737 chars, depth 5, 40 keys 10,739, depth 14, 397 keys 14.6×

Generation succeeds — no recursion error — but emits no $defs and no $ref. FlowNodeSchema is inlined in full at every region key, so loop.body arrives as the entire node/edge definition:

"body": { "type": "object", "properties": { "nodes": { "minItems": 1, "type": "array",
  "items": { "type": "object", "properties": {
    "id": {}, "type": {}, "label": {},
    "config": { "type": "object", "propertyNames": {}, "additionalProperties": {} },
    "connectorConfig": {} } } } } }

versus the published nodes: { type: 'array' } with no items — opaque, because a loop body is edited on the canvas, not in a property form.

Making the generated output usable means a projection that prunes ~90% of it back off (201 → 25 keys), at which point "one source" is a fiction: you maintain the Zod, the projection rules, and a check that the projection still does the right thing — three things instead of two.

This corrects the premise #4045 opened with (and my own first analysis on it), which treated these three as redundant copies awaiting de-duplication. They are a second artifact with a different job, and the gaps are deliberate. Full data in the issue.

So the divergence is declared rather than erased

A DELIBERATELY_SHALLOW ledger names each shallow key with its reason (loop.body, parallel.branches, try_catch.try, try_catch.catch), and every entry must name a key both sides still declare — so it cannot rot into a reference to something renamed away. Same bidirectional discipline as the #4040 expression ledger.

One deliberate limitation, stated in the file

This compares key sets, not generated shapes. Generating needs z.toJSONSchema, and service-automation does not depend on zod — only @objectstack/spec does. Adding that edge to the package graph to run a test is not worth it when schema.shape yields the key set without it, and the key set is where the drift that matters lives. The depth divergence is measured once in the table above, and its form half is pinned by config-schemas.test.ts. The test file says all of this, so the next reader does not assume it was forgotten.

Verified to bite in all three directions

An assertion that never fails is decoration:

# Zod gains a key the form does not offer
× 'loop': the form offers exactly the keys the Zod accepts
AssertionError: loop: accepted by the Zod but absent from the designer form:
  expected [ 'brandNewRegionKey' ] to deeply equal []

# the form gains a key the Zod rejects
× 'loop': the form offers exactly the keys the Zod accepts   (inverse set)

# a ledger entry points at a renamed key
× every ledger entry names a key both sides actually declare
AssertionError: loop.bodyRenamedAway: not on the form any more

Test plan

Check Result
@objectstack/service-automation ✅ 457 passed (5 new)
ESLint on the new file ✅ clean
spec check:api-surface / check:authorable-surface / check:docs ✅ PASS

The negative controls rebuilt spec, which regenerates authorable-surface.json — checked back to clean and the gates re-run, so no generated artifact rides along in this diff.

Empty changeset — tests only, nothing releases.

Remaining in #4045

  • Flat half of step Cscreen / CRUD / http / notify have no region keys, so the single-Zod-source route stays viable there; the one thing needing explicit handling is additionalProperties on the 7 keyValue slots (z.record() emits {}, not the load-bearing true; .meta({ additionalProperties: true }) overrides it, and objectui accepts {} anyway per json-schema-to-fields.ts:456).
  • Step 3b — tightening the feat(automation,formula): warn on undeclared flow-node config keys (#4045) #4059 unknown-key warning into an error still needs a release of real-world distribution data first.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QoA8AV99Ss1RRkLLAYmDzq


Generated by Claude Code

…heir Zod (#4045)

`loop` / `parallel` / `try_catch` each carry two descriptions of the same config — a
hand-written `configSchema` on the descriptor (drives the Studio form) and a Zod
schema in `control-flow.zod.ts` (validates the value) — and nothing compared them.
#4064 pinned the shape of the first; this pins the relationship, so neither side can
gain or lose a key without the other noticing.

Asserts both directions. A key the Zod accepts but the form omits is #3528's failure
inverted: a real config key with no way to author it in Studio, discoverable only by
reading the executor. A key the form offers but the Zod rejects means the author
fills in a field whose value is then dropped.

The two sources are deliberately NOT merged, and that is now measured rather than
assumed. Generating the form from the Zod emits 9–17× more schema at +9 levels of
depth (loop 597 → 5,537 chars / 25 → 201 keys; parallel 294 → 5,112; try_catch
737 → 10,739) with NO $defs and NO $ref — FlowNodeSchema is inlined in full at every
region key, so a loop body would reach the designer as the entire node/edge
definition instead of the opaque array a canvas-edited sub-graph needs. A projection
pruning ~90% back off leaves three things to maintain instead of two, so "one source"
would be a fiction. This corrects the premise #4045 opened with.

Since the divergence is legitimate, what is enforced is that it stays DECLARED: a
`DELIBERATELY_SHALLOW` ledger names each shallow key with a reason, and every entry
must name a key both sides still declare — so it cannot rot into a reference to
something renamed away.

Compares key sets, not generated shapes, on purpose: generating needs
`z.toJSONSchema` and `service-automation` does not depend on `zod` (only spec does).
Adding that edge to the package graph to run a test is not worth it when
`schema.shape` gives the key set without it, and the key set is where the drift that
matters lives. The file says so, so the next reader does not think it was forgotten.

Verified to bite in all three directions, not merely to pass:
- a Zod key the form omits → `loop: accepted by the Zod but absent from the designer
  form: expected [ 'brandNewRegionKey' ] to deeply equal []`
- a form key the Zod rejects → same test red on the inverse set
- a ledger entry pointing at a renamed key → `loop.bodyRenamedAway: not on the form
  any more`

Test plan: service-automation 457 passed (5 new), ESLint clean, and spec's
api-surface / authorable-surface / docs gates still pass (the negative controls
rebuilt spec, so the generated artifacts were checked back to clean).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QoA8AV99Ss1RRkLLAYmDzq
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Jul 30, 2026 8:37am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/m labels Jul 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): packages/services.

6 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/automation/webhooks.mdx (via packages/services)
  • content/docs/kernel/runtime-services/audit-service.mdx (via packages/services)
  • content/docs/kernel/runtime-services/index.mdx (via packages/services)
  • content/docs/kernel/runtime-services/settings-service.mdx (via packages/services)
  • content/docs/plugins/packages.mdx (via packages/services)
  • content/docs/protocol/kernel/i18n-standard.mdx (via packages/services)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review July 30, 2026 08:53
@os-zhuang
os-zhuang merged commit 46e86ba into main Jul 30, 2026
17 checks passed
@os-zhuang
os-zhuang deleted the claude/console-screen-flow-submit-jfpjy4 branch July 30, 2026 08:53
os-zhuang added a commit that referenced this pull request Jul 30, 2026
…4091)

`affected-docs.mjs` derived changed-package roots from every file under `packages/`,
tests included. A test observes behaviour rather than defining it, so it cannot make an
implementation-accuracy doc stale — yet every tests-only PR lit up its packages' whole
doc set. Three in a row did it (#4064, #4078 and one before), each flagging 6
packages/services docs for a diff with no production code.

This is the one place the mapper's deliberate over-inclusion actively hurt: a category
that is ALWAYS false teaches the reader to skip the comment, and then it fails on the PR
where it is right.

- Test files are dropped before the roots are derived: *.test.* / *.spec.* at any depth
  (covering .integration.test.ts and .conformance.test.ts) plus __tests__ / __mocks__ /
  __fixtures__.
- The narrowing is NOT silent — the excluded count rides the summary line and
  `testFilesSkipped` in --json.
- `--self-test` pins the matcher (the check-error-code-casing / check-route-envelope
  convention), run by the workflow before it computes the mapping.
- The workflow also triggers on the mapper's own paths, so editing it runs its own guard.
  CI surfaced that gap: on the first push the drift job never ran at all, which meant the
  self-test had been added but was structurally unreachable by the change most likely to
  break it.

Verified against real history: 46e86ba (tests only) → 0 docs, was 6; 4965bfa
(engine.ts + formula) → the same 11 docs as before; --all unchanged at 178. Swapping the
anchored match for a naive path.includes('test') fails 7 self-test cases, including
control-flow.zod.ts and latest.ts being swallowed as tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants