Conversation
…titution output shapes (#3796) runner-output-contract.yaml (1.1.0 → 1.2.0): - Adds capture_path_not_resolvable and unresolved_substitution to the validation_result.check enum with defined expected/actual/json_pointer semantics. Both codes belong in validation_result so the "failed steps MUST include at least one validation result with passed: false" invariant holds without a contract amendment. - Documents null/empty-string as equally non-resolvable for captures. - Adds pre-wire null carve-outs on request, response, and json_pointer for unresolved_substitution (no request was sent; no response exists). - Adds run_summary note that capture failures contribute to steps_failed, not steps_skipped. storyboard-schema.yaml: - Strengthens context_outputs runner-behavior note to cover absent/null/"" cases and cross-references runner-output-contract.yaml for output shapes. - Expands capture_path_not_resolvable and unresolved_substitution grading code descriptions with the three non-resolvable value cases and output shape cross-references. signal-marketplace/index.yaml: - Adds explicit field_present guard on signals[0] and signals[1] before deeper-path checks so empty/short-array responses fail the capturing step (not downstream substitution steps). https://claude.ai/code/session_01Wxmx2xnsUoMwMPY4e5Figj
This was referenced May 1, 2026
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3796
Summary
Two grading codes (
capture_path_not_resolvable,unresolved_substitution) were defined instoryboard-schema.yamlbut had no corresponding output shape inrunner-output-contract.yaml. Without normative output shapes, runners had no spec to implement against — and in practice silently swallowed context-extraction failures, reporting them as downstream substitution skips rather than as failures on the capturing step. The diagnostic pointed at the wrong place.Root symptom (from #3796):
signal_marketplacestoryboard, stepsearch_by_specreturns a shape-valid but emptysignals: []response. The runner doesn't failsearch_by_spec; insteadactivate_signalsteps appear asSkipped: unresolved context variables: first_signal_agent_segment_id, first_signal_pricing_option_id, making it look like the activation handler is missing when the signals handler broke the contract.Changes
runner-output-contract.yaml(v1.1.0 → v1.2.0)capture_path_not_resolvableandunresolved_substitutionto thevalidation_result.checkenum. Both appear invalidations[]so the existing "failed steps MUST include at least one validation result withpassed: false" invariant is satisfied without a contract amendment.expected/actual/json_pointersemantics for each code.""are equally non-resolvable as an absent path for captures — capturing a null value produces fabricated downstream state.request,response, andjson_pointerforunresolved_substitution(no request was sent; no response exists).run_summarynote that capture failures contribute tosteps_failed, notsteps_skipped.storyboard-schema.yamlcontext_outputsrunner-behavior note to cover absent/null/""and cross-references runner-output-contract.yaml for the output shapes.capture_path_not_resolvableandunresolved_substitutiongrading-code descriptions with the three non-resolvable value cases and output shape pointers.signal-marketplace/index.yamlfield_presentguard onsignals[0]andsignals[1]before deeper-path checks insearch_by_spec, so empty/short-array responses fail the capturing step rather than cascading as downstream substitution skips.Non-breaking justification: Additive only — new
checkenum values that conformant runners currently do not emit; new inline semantics clauses; no existing check value behavior changed. Runners not yet implementing these codes remain conformant for their current output; the contract now gives them a target to implement against.Pre-PR review:
request/response/json_pointerforunresolved_substitution); one nit addressed (signals[1]guard added)check-enum placement correct given thepassed: falseinvariant; null/"" semantics correctly close the loophole; v1.2.0 minor bump correctly scopedSibling-repo fix needed: The runner bug (silently swallowing capture failures instead of emitting
capture_path_not_resolvable) lives in adcp-client. This PR ships the normative spec the runner needs; the adcp-client runner implementation is the load-bearing follow-up that stops the symptom described in #3796.Session: https://claude.ai/code/session_01Wxmx2xnsUoMwMPY4e5Figj
Generated by Claude Code