docs(workflows): clarify acceptance requiredEvidence type, limits, and usage (#207) - #223
Conversation
agnitum2009
left a comment
There was a problem hiding this comment.
APPROVE — independent verification against cc1b174; all claims confirmed.
Local verification (fetched the branch and ran the stated test plan):
bun run check-> exit 0bun run test-> 965 pass (935 Node + 30 Vitest), matching the PR description exactly- New acceptance tests: 6/6 in
tests/extensions/workflows/acceptance.test.ts
Docs <-> implementation cross-check (extensions/workflows/acceptance.ts): every documented constraint matches the code — criteria 1-32 (MAX_CRITERIA=32, zero rejected), description 1-500 chars, requiredEvidence must be a string[], up to 16 labels, each label non-blank and up to 120 chars ("invalid evidence label"), exact-match evidence via result.evidence.includes(required). The new tests assert the actual error messages, so they are genuine regression protection.
Diff is docs + tests only — no runtime behavior change, consistent with the stated impact.
Non-blocking nits: the blank-label rule could be named in README, and description length is measured after trim — both cosmetic.
|
This focused Workflow documentation and regression-test PR remains open under #277. Its next gate is an update to current main, exact-head review of the requiredEvidence behavior contract, and fresh CI. No merge action is taken by this triage. |
cc1b174 to
dcb1b3e
Compare
tt-a1i
left a comment
There was a problem hiding this comment.
Independent exact-head review: solves the requiredEvidence string-array documentation gap without changing runtime semantics; documents current criteria/label limits and exact matching; adds focused rejection coverage. Focused acceptance tests 6/6, bun run check, diff-check, and all three GitHub CI jobs passed. No findings.
…d usage (openpi-dev#207) (openpi-dev#223) (cherry picked from commit a15a0d8)
Problem
Workflow API callers can easily mistake
acceptance.criteria[].requiredEvidencefor a prose description field instead of a concise string array (string[]). Passing a scalar string currently fails invocation admission with an error without clear documentation guidance and lacks direct regression test coverage in acceptance unit tests (Fixes #207).Value
Clarifying the parameter type, character/length limits (1–32 criteria, ≤16 evidence labels, ≤120 chars per label), and exact match requirements prevents caller confusion and provides explicit contract documentation and regression protection.
Approach
skills/workflows/REFERENCE.mdto definerequiredEvidence?: string[], specify the 1–32 criteria and 16-label limits, and clarify exact matching against returnedacceptance.criteria[].evidence.README.mdwith an explicit acceptance example and field/constraint explanations.tests/extensions/workflows/acceptance.test.tsverifying that passing scalar strings, >16 labels, blank strings, or oversized labels fail with appropriate errors.Validation
bun run check- passed (format, lint, typecheck)bun run test- 965 tests passed (935 Node + 30 Vitest), 0 failuresgit diff --check- passedImpact
requiredEvidencetyping and examples.