Skip to content

fix(spec): reject undeliverable IOA webhooks - #386

Draft
nerdsane wants to merge 4 commits into
mainfrom
codex/arn-227-ioa-webhook-runtime
Draft

fix(spec): reject undeliverable IOA webhooks#386
nerdsane wants to merge 4 commits into
mainfrom
codex/arn-227-ioa-webhook-runtime

Conversation

@nerdsane

@nerdsane nerdsane commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • reject action-owned and legacy IOA webhook declarations before trigger synthesis, verification, JIT construction, or registration
  • preserve entity, WASM, adapter, registered custom integrations, generic custom effects, operator webhooks.toml, and the directly configured platform engine API
  • document the journaled intent, replay/retry, stable delivery ID, authorization, and durable outcome gates required before webhook syntax can be accepted again
  • correct agent/MCP guidance and remove invalid no-op webhook declarations from repository fixtures

Regression strategy

  • ADR-0171 landed first after checking its number against main and open PRs
  • commit 2549a376 contains only failing behavioral/durability regressions for action-trigger, explicit legacy, and defaulted legacy webhook forms
  • commit ee14b001 is the reviewed GREEN boundary and compatibility cleanup

Validation status

  • temper-spec: 269/269 passed
  • directly configured platform integration engine: 9/9 passed with loopback access
  • affected app/fixture directories: L0-L3 and composite verification passed in the pre-commit hook
  • rustfmt, strict workspace Clippy, and readability ratchet passed in the pre-push hook
  • code-quality review: PASS
  • DST review: PASS
  • full workspace test execution reached an unchanged temper-actor-runtime test target, where all five tests timed out creating Postgres containers because Docker was unresponsive (CreateContainer(RequestTimeoutError)); final Docker-dependent rerun remains required before shippable status

Linear: https://linear.app/arni-build/issue/ARN-227/bug-ioa-webhook-integrations-are-accepted-by-specs-but-never-execute

Greptile Summary

This PR eliminates a silent false promise: IOA webhook declarations ([[action.triggers]] kind = \"webhook\" and legacy [[integration]] type = \"webhook\") were accepted, verified, and installed but never dispatched — the production runtime only executes wasm and adapter integration types. ADR-0171 formalizes the rejection and defines the six-gate acceptance contract for a future durable delivery runtime.

  • Parser enforcement: validate now rejects both declaration forms before synthesis, verification, JIT construction, or registration; the omitted-type default (which historically resolved to \"webhook\") is also caught, and expand_external_action_triggers adds a redundant defensive check as a second layer.
  • Fixture and documentation cleanup: heartbeat_run.ioa.toml, subscription.ioa.toml, and the temper-platform integration test fixtures remove or replace all webhook declarations; AGENT_GUIDE.md, PAPER.md, SKILL.md, protocol.rs, and both superseded ADRs are updated consistently.
  • Preserved capabilities: operator-level webhooks.toml trajectory subscriptions, the directly configured temper-platform integration engine (with its retry/dead-letter API), entity triggers, WASM triggers, and adapter triggers are all unchanged.

Confidence Score: 5/5

Safe to merge: removes a false promise (silent accept + no dispatch) with no working capability dropped, and all three affected declaration paths are covered by targeted regression tests.

The change is a pure narrowing of what the parser accepts. Every previously working runtime path (entity triggers, WASM, adapters, operator webhooks, and the standalone platform integration engine) is explicitly preserved and tested. The two new rejection sites fire before any state-mutating step, and the six test cases covering explicit webhook type, omitted type, inline action trigger, field-presence ordering, and expansion prevention leave no gap in the enforcement contract.

No files require special attention. The defensive webhook check inside expand_external_action_triggers is unreachable (validate fires first), but it is harmless and serves as a guard against future refactoring that might reorder the two calls.

Important Files Changed

Filename Overview
crates/temper-spec/src/automaton/parser.rs Core rejection logic: validate now catches both TriggerKind::Webhook in action triggers and integration_type == webhook in integration declarations before expansion; expand_external_action_triggers adds a redundant defensive check. All three paths are well-covered by tests.
crates/temper-spec/src/automaton/parser_integrations_test.rs Old webhook-acceptance tests replaced with explicit rejection tests; adds coverage for explicit type = webhook, omitted type (defaulted webhook), and the inline trigger form; existing supported integration kinds still verified.
crates/temper-spec/src/automaton/parser_triggers_test.rs Renames and rewrites webhook-kind tests to assert rejection; adds test proving rejection precedes field-presence validation; expansion and entity-kind regression tests unchanged.
crates/temper-spec/src/automaton/types.rs Doc-only changes; TriggerKind::Webhook and its source fields (url, method, headers, body_template) retained for deserialization so users receive the actionable ADR-0171 error rather than an unknown-field parse failure.
crates/temper-platform/tests/integration_engine.rs Test fixture integrations switched from type = webhook to type = wasm with module fields; registry test now constructs configs directly (no longer relies on parsed IOA webhook declarations as an ingestion path).
docs/adrs/0171-reject-undeliverable-ioa-webhooks.md New ADR accepted and complete: context, decision, rollout plan, readiness gates, consequences (including DST compliance), and the six-item gate for future webhook re-acceptance all present. Status correctly set to Accepted.
skills/temper-agent/SKILL.md http_fetch WASM trigger example corrected to place url/method under [action.triggers.config] instead of as top-level trigger fields; webhook section updated from parse-only to rejected; kind enum corrected from webhook to adapter.
os-apps/agent-orchestration/specs/heartbeat_run.ioa.toml Removes two no-op [[integration]] type = webhook declarations (notify_completed, notify_failed) that would have failed validation after this change.
test-fixtures/specs/subscription.ioa.toml Removes two legacy [[integration]] type = webhook declarations (billing_webhook, dunning_notice) that were never delivered and would now fail validation.
crates/temper-server/src/registry/relations.rs Doc-only: synthesize_action_trigger_reaction comment updated to reflect that webhook triggers cannot reach registry synthesis from a validated IOA spec.
crates/temper-mcp/src/protocol.rs MCP system prompt updated to remove webhook from the accepted trigger kind list and clarify that both webhook declaration forms are rejected.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[parse_toml_to_automaton] --> B[validate]
    B --> C{integration_type == webhook?}
    C -- Yes --> ERR1[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED]
    C -- No --> D[validate_action_triggers]
    D --> E{trigger.kind == Webhook?}
    E -- Yes --> ERR2[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED]
    E -- No --> F[wire_state_timeout_from_states]
    F --> G[expand_external_action_triggers]
    G --> H{trigger.kind == Webhook? defensive}
    H -- Yes --> ERR3[AutomatonParseError dead path validate fired first]
    H -- No --> I{kind == Wasm or Adapter?}
    I -- Yes --> J[Synthesize Integration and Trigger Effect]
    I -- No --> K[Skip Entity or other]
    J --> L[check_liveness_coverage]
    K --> L
    L --> M[Automaton accepted no webhook declarations possible]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[parse_toml_to_automaton] --> B[validate]
    B --> C{integration_type == webhook?}
    C -- Yes --> ERR1[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED]
    C -- No --> D[validate_action_triggers]
    D --> E{trigger.kind == Webhook?}
    E -- Yes --> ERR2[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED]
    E -- No --> F[wire_state_timeout_from_states]
    F --> G[expand_external_action_triggers]
    G --> H{trigger.kind == Webhook? defensive}
    H -- Yes --> ERR3[AutomatonParseError dead path validate fired first]
    H -- No --> I{kind == Wasm or Adapter?}
    I -- Yes --> J[Synthesize Integration and Trigger Effect]
    I -- No --> K[Skip Entity or other]
    J --> L[check_liveness_coverage]
    K --> L
    L --> M[Automaton accepted no webhook declarations possible]
Loading

Reviews (2): Last reviewed commit: "docs: accept ADR 0171" | Re-trigger Greptile

@nerdsane

Copy link
Copy Markdown
Owner Author

RED evidence · commit 2549a37

The regressions fail because both IOA webhook forms currently parse successfully and materialize integration_type = "webhook"; the action-trigger form also synthesizes __trigger__:ConfirmOrder:notify_fulfillment.

$ env CARGO_TARGET_DIR=/tmp/temper-arn227-target cargo test -p temper-spec rejected_until_delivery_is_durable -- --nocapture
running 2 tests
... webhook_action_trigger_is_rejected_until_delivery_is_durable ... FAILED
... legacy_webhook_integration_is_rejected_until_delivery_is_durable ... FAILED
test result: FAILED. 0 passed; 2 failed; 268 filtered out
$ env CARGO_TARGET_DIR=/tmp/temper-arn227-target cargo test -p temper-spec defaulted_legacy_webhook_integration_is_also_rejected -- --nocapture
running 1 test
... defaulted_legacy_webhook_integration_is_also_rejected ... FAILED
test result: FAILED. 0 passed; 1 failed; 269 filtered out

@nerdsane

Copy link
Copy Markdown
Owner Author

Fresh GPT-5.6 review of the GitHub PR metadata and complete diff.

No actionable findings.

Evidence:

  • Correctness/durability: parser validation rejects both action-trigger webhooks and explicit/defaulted legacy webhook integrations before synthesis, JIT, or runtime; no best-effort HTTP dispatcher was introduced.
  • Regression safety: entity, WASM, adapter, and unpaired generic trigger-effect paths retain regression coverage. The operator webhook subsystem and standalone IntegrationEngine implementation are untouched, while direct engine configuration remains tested.
  • Tests/docs: regressions cover both webhook declaration forms, the omitted legacy type, pre-expansion rejection, and supported integration parsing/expansion. Documentation states the crash-loss boundary and the journaled-delivery gate for future support. Compile/lint, DST patterns, spec verification, instrumentation hygiene, and completed platform checks were green at review time; some CI jobs were still running.

Verdict: PASS

@nerdsane

Copy link
Copy Markdown
Owner Author

@greptile review

Comment thread docs/adrs/0171-reject-undeliverable-ioa-webhooks.md
@nerdsane

Copy link
Copy Markdown
Owner Author

Fresh follow-up review at HEAD 9c54061.

No actionable findings.

Evidence:

  • The prior ADR finding is fixed: the current GitHub diff records ADR-0171 as Status: Accepted, consistent with the parser behavior shipping in this PR.
  • The current patch still rejects action-trigger and explicit/defaulted legacy IOA webhooks before synthesis and runtime, without introducing a partial delivery path.
  • Supported entity, WASM, adapter, registered custom-effect, operator webhook, and directly configured IntegrationEngine behavior remains preserved by the implementation boundary and regression coverage.
  • The new HEAD adds only the ADR status correction and introduces no new code or test risk. Completed HEAD checks are green; some CI jobs remain in progress.

Verdict: PASS

@nerdsane

Copy link
Copy Markdown
Owner Author

@greptile review final head 9c54061

@nerdsane

Copy link
Copy Markdown
Owner Author

Live local E2E evidence · final HEAD 9c54061b

The pre-fix RED behavior is captured in the isolated RED comment: both outbound webhook forms parsed successfully and materialized an undeliverable integration_type = "webhook".

The final build was started with an app containing a real action-owned webhook target (http://127.0.0.1:39127/fulfillment):

$ env TURSO_URL=file:/tmp/temper-arn227-e2e.db /tmp/temper-arn227-target/debug/temper serve --port 39126 --no-observe --specs-dir .progress/arn-227-e2e --tenant arn227-e2e
OTEL export configured: endpoint=https://logfire-us.pydantic.dev source=LOGFIRE_TOKEN logfire_auth=true
  Storage: turso (file:/tmp/temper-arn227-e2e.db)
  Loading app: arn227-e2e from .progress/arn-227-e2e
Error: failed to parse IOA spec for Order

Caused by:
    validation error: trigger 'notify_fulfillment' on action 'ConfirmOrder': outbound IOA webhooks are unsupported until durable delivery is available
[exit code: 1]

The process exits during app registration, before the server binds and before any source action can commit:

$ curl --silent --show-error --max-time 2 http://127.0.0.1:39126/health
curl: (7) Failed to connect to 127.0.0.1 port 39126 after 0 ms: Couldn't connect to server
[exit code: 7]

Result: the previously accepted-and-dropped webhook can no longer install as healthy; rejection is stable, explicit, and occurs before runtime delivery is possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants