Skip to content

fix(compliance): define capture_path_not_resolvable / unresolved_substitution output shapes#3798

Closed
bokelley wants to merge 1 commit intomainfrom
claude/issue-3796-capture-path-not-resolvable-runner-contract
Closed

fix(compliance): define capture_path_not_resolvable / unresolved_substitution output shapes#3798
bokelley wants to merge 1 commit intomainfrom
claude/issue-3796-capture-path-not-resolvable-runner-contract

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

@bokelley bokelley commented May 1, 2026

Closes #3796

Summary

Two grading codes (capture_path_not_resolvable, unresolved_substitution) were defined in storyboard-schema.yaml but had no corresponding output shape in runner-output-contract.yaml. Without normative output shapes, runners had no spec to implement against — and in practice silently swallowed context-extraction failures, reporting them as downstream substitution skips rather than as failures on the capturing step. The diagnostic pointed at the wrong place.

Root symptom (from #3796): signal_marketplace storyboard, step search_by_spec returns a shape-valid but empty signals: [] response. The runner doesn't fail search_by_spec; instead activate_signal steps appear as Skipped: unresolved context variables: first_signal_agent_segment_id, first_signal_pricing_option_id, making it look like the activation handler is missing when the signals handler broke the contract.

Changes

runner-output-contract.yaml (v1.1.0 → v1.2.0)

  • Adds capture_path_not_resolvable and unresolved_substitution to the validation_result.check enum. Both appear in validations[] so the existing "failed steps MUST include at least one validation result with passed: false" invariant is satisfied without a contract amendment.
  • Documents expected/actual/json_pointer semantics for each code.
  • Clarifies that null and "" are equally non-resolvable as an absent path for captures — capturing a null value produces fabricated downstream state.
  • Adds pre-wire null carve-outs on request, response, and json_pointer for unresolved_substitution (no request was sent; no response exists).
  • Adds run_summary note that capture failures contribute to steps_failed, not steps_skipped.

storyboard-schema.yaml

  • Strengthens the context_outputs runner-behavior note to cover absent/null/"" and cross-references runner-output-contract.yaml for the output shapes.
  • Expands capture_path_not_resolvable and unresolved_substitution grading-code descriptions with the three non-resolvable value cases and output shape pointers.

signal-marketplace/index.yaml

  • Adds explicit field_present guard on signals[0] and signals[1] before deeper-path checks in search_by_spec, so empty/short-array responses fail the capturing step rather than cascading as downstream substitution skips.

Non-breaking justification: Additive only — new check enum values that conformant runners currently do not emit; new inline semantics clauses; no existing check value behavior changed. Runners not yet implementing these codes remain conformant for their current output; the contract now gives them a target to implement against.

Pre-PR review:

  • code-reviewer: approved — fixed one blocker (pre-wire null carve-outs on request/response/json_pointer for unresolved_substitution); one nit addressed (signals[1] guard added)
  • ad-tech-protocol-expert: approved — check-enum placement correct given the passed: false invariant; null/"" semantics correctly close the loophole; v1.2.0 minor bump correctly scoped

Sibling-repo fix needed: The runner bug (silently swallowing capture failures instead of emitting capture_path_not_resolvable) lives in adcp-client. This PR ships the normative spec the runner needs; the adcp-client runner implementation is the load-bearing follow-up that stops the symptom described in #3796.

Triage-managed PR. This bot does not currently iterate on
review comments or PR conversation threads (only on the source
issue). To unblock:

  • Push fixup commits directly: gh pr checkout <num>
    fix → push.
  • Or re-trigger: comment /triage execute on the source
    issue.

See #3121
for context.

Session: https://claude.ai/code/session_01Wxmx2xnsUoMwMPY4e5Figj


Generated by Claude Code

…titution output shapes (#3796)

runner-output-contract.yaml (1.1.0 → 1.2.0):
- Adds capture_path_not_resolvable and unresolved_substitution to the
  validation_result.check enum with defined expected/actual/json_pointer
  semantics. Both codes belong in validation_result so the "failed steps
  MUST include at least one validation result with passed: false" invariant
  holds without a contract amendment.
- Documents null/empty-string as equally non-resolvable for captures.
- Adds pre-wire null carve-outs on request, response, and json_pointer
  for unresolved_substitution (no request was sent; no response exists).
- Adds run_summary note that capture failures contribute to steps_failed,
  not steps_skipped.

storyboard-schema.yaml:
- Strengthens context_outputs runner-behavior note to cover absent/null/""
  cases and cross-references runner-output-contract.yaml for output shapes.
- Expands capture_path_not_resolvable and unresolved_substitution grading
  code descriptions with the three non-resolvable value cases and output
  shape cross-references.

signal-marketplace/index.yaml:
- Adds explicit field_present guard on signals[0] and signals[1] before
  deeper-path checks so empty/short-array responses fail the capturing step
  (not downstream substitution steps).

https://claude.ai/code/session_01Wxmx2xnsUoMwMPY4e5Figj
@bokelley
Copy link
Copy Markdown
Contributor Author

bokelley commented May 2, 2026

Closing in favor of the unified landing in #3813 — spec + runner ship together so the user-visible symptom in #3796 actually gets fixed. The signals[0]/signals[1] field_present workarounds become unnecessary once the runner emits capture_path_not_resolvable against the capturing step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-triaged Issue has been triaged by the Claude Code triage routine. Remove to re-triage.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Storyboard runner: stateful step cascade silently skips downstream steps when prior response is shape-valid but missing extractor-required fields

2 participants