docs(liveness): re-verify the last ten preview-only live claims — 8 of 10 were wrong (#3686) - #3711
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 104 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…f 10 were wrong (#3686) Closes the sweep opened in #3685. Final tally across all 13 properties the 2026-06 pass marked `live` on preview-renderer evidence: 3 stand, 10 were wrong — a 77% error rate for that standard. Corrected to dead + authorWarn (with actionable hints): - action.shortcut / action.bulkEnabled — registered into ActionEngine, but getShortcuts()/handleShortcut()/getBulkActions()/executeBulk() have no non-test caller and no keydown listener feeds them. Bulk toolbars are driven by the LIST VIEW's bulkActions/bulkActionDefs instead. - flow.active — deprecated no-op; `status` is what gates binding/execution. Sharper than inert: the spec default is false while an unset flow runs, so the key reads as if it disabled the flow. - skill.triggerPhrases — a runtime path reads it, but only to hand it back to API clients. Phrases are never matched against user messages; activation is triggerConditions + the agent's skills[] + explicit /skill-name pinning. - tool.category / active / builtIn — not part of AIToolDefinition; the tool set reaches the LLM as name/description/parameters only. Contrast agent.active and skill.active, which ARE enforced. - tool.requiresConfirmation — SAFETY-shaped and unenforced on every path (LLM tool set, ToolRegistry.execute, the REST execute route, the MCP bridge). Every real requiresConfirmation read is action.ai.* — a different, live property. ADR-0033 already resolved to delete this placeholder. Kept live, evidence corrected to the real reader: - action.execute — ActionRunner.ts:704 + the spec's execute->target transform. Records a live divergence: the transform prefers `target`, ActionRunner prefers `execute`, so an action declaring both runs different code on the two sides. - flow.status — engine.ts:1374-1382 gates binding AND execution since 497bda8. The file-level _note claiming "status/active gate nothing" was true when written and became false a month later; rewritten. README: count tables synced (action 33/1/2, flow 26/-/5, skill 8/-/2, tool 5/1/5) and the methodology section extended with the final tally, the two failure directions (over- AND under-stating liveness), and the three search traps that produced false negatives during this work — including that macOS `git grep -E` silently ignores `\b`. check:liveness green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
xuyushun441-sys
force-pushed
the
fix/liveness-ledger-ten-preview-claims
branch
from
July 27, 2026 15:46
0112d33 to
829d837
Compare
github-actions
Bot
added
documentation
This was referenced Jul 27, 2026
[P2] action
execute vs target: spec and objectui resolve the precedence in OPPOSITE directions
#3713
Closed
os-zhuang
added a commit
that referenced
this pull request
Jul 28, 2026
… provide (#3715) (#3740) The flag is read by no execution path (LLM tool set, ToolRegistry.execute, the REST execute route, the MCP bridge — verified in #3711), while the authoring surface actively taught reliance on it: a form section titled "Access & safety" with helpText "Ask user to approve before executing (for destructive actions)", plus SKILL.md / MCP_GUIDE / README all recommending it for destructive work. Owner call: DEFER the prune-or-wire decision (#3715), stop the false promise now. The shape is likely needed once side-effect tools exist — ToolCategory already anticipates `action` (send email / create record), `integration` (external API) and `flow`, none of which the ADR-0033 draft/publish gate covers; that ADR's "the draft is the approval gate" reasoning held when the only tools were metadata mutators. - spec describe: [EXPERIMENTAL — not enforced] + pointer to the real gate - form: section renamed "Declarative metadata (not enforced)"; both fields (this + the already-dead `permissions`) name the enforced alternative - SKILL.md / MCP_GUIDE.md / README.md: point at action-level ai.requiresConfirmation + the approval queue, and note that AI metadata edits are already gated by draft/publish - ledger note records the deferral - regenerated docs + the four i18n bundles (diff verified: only the tool form's label/description/helpText keys moved) No behaviour change. 6710 spec tests; liveness/docs/api-surface/skill-docs/ skill-examples/i18n/role-word all green. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the preview-claim sweep opened in #3685. Ledger-only; releases nothing.
Final tally: the preview-renderer standard was wrong 77% of the time
Of the 13 properties a 2026-06 pass marked
liveciting only ametadata-admin/previews/*Preview.tsxpanel, 10 were wrong.Corrected →
dead+authorWarnaction.shortcutActionEngine.shortcuts[], butgetShortcuts()/handleShortcut()have no non-test caller and no keydown listener feeds them; objectui's real keyboard stack never touches ActionEngineaction.bulkEnabledgetBulkActions()/executeBulk()uncalled. The multi-select toolbar is driven by the list view'sbulkActions/bulkActionDefsflow.activefalsewhile an unset flow runs, so the key reads as if it disabled the flowskill.triggerPhrasestriggerConditions+ the agent'sskills[]+ explicit/skill-namepinningtool.category/active/builtInAIToolDefinition; tools reach the LLM as name/description/parameters only.tool.activeis the sharp one —agent.activeandskill.activeare enforced, so the inconsistency is invisible to an authortool.requiresConfirmationKept
live, evidence corrected to the real readeraction.execute—ActionRunner.ts:704+ the spec'sexecute → targettransform.targetwhen both are set,ActionRunnerdoesexecute || target. An action declaring both runs different code client- vs server-side.flow.status—engine.ts:1374-1382gates binding and execution. The file-level_noteclaiming "status/active gate nothing" was true when written and falsified a month later by497bda853; rewritten.tool.requiresConfirmationdeserves its own noteNothing pauses for confirmation on this flag — not the LLM tool set (
vercel-adapter.ts:263-272), notToolRegistry.execute, notPOST /ai/tools/:name/execute, not the MCP bridge (which uses a hardcoded destructive-name list). Every realrequiresConfirmationread in all three repos isaction.ai.requiresConfirmation— a different, genuinely live property.What makes it worse than inert: the authoring surface teaches reliance on it.
tool.form.ts:36-42puts it in a section titled "Access & safety" with helpText "Ask user to approve before executing (for destructive actions)", andskills/objectstack-ai/SKILL.md:547tells authors to use it for destructive operations.ADR-0033 already resolved this (
:31,:60): it records the flag as never-enforced and states "We delete the unenforcedrequiresConfirmationplaceholder" — the draft/publish workspace is the real approval gate. That deletion was never executed. This PR marks it honestly; the prune is proposed as a follow-up (same disposition the owner just chose forskill.permissionsin #3704).Also in this PR
Count tables synced (action
33/1/2, flow26/–/5, skill8/–/2, tool5/1/5) and the methodology section extended with:flow.statuswas understated, because code moved under a correct-at-the-time note;git grep -Esilently ignores\b(git grep -cE "\.active\b"returns nothing on a file with three.activehits);check:livenessgreen.Refs #3686, #3685, #1878.
🤖 Generated with Claude Code