WebhookTrigger: provider-agnostic template verifier#1189
Merged
Conversation
Replaces the v1 HMAC shorthand with a declarative template engine. The
constraint is strict and honored at every layer: *no provider names live
anywhere in DM core*. All HMAC behaviour is driven by config on the flow,
either hand-written or expanded from a filter-registered preset.
Core architecture
- WebhookVerifier (new): single static `verify()` driven by a `signed_template`
grammar ({body} {timestamp} {id} {url} {header:X} {param:X}) + four extract
kinds (raw / prefix / kv_pairs / regex) + three signature encodings (hex /
base64 / base64url) + optional replay-window enforcement.
- WebhookVerificationResult (new): structured result with 11 outcome codes
and diagnostic fields (secret_id, timestamp, skew_seconds, detail).
- WebhookAuthResolver (new): owns the single entry point that converts a
flow's scheduling_config into a canonical verifier config. Also contains
the ONLY place in core that knows about the v1 field enum — it migrates
legacy flows silently on first read, then deletes the legacy fields.
Zero-hardcoding guarantees
- No `'sha256=hex' | 'hex' | 'base64'` enum survives outside the migration
helper. The stored shape for HMAC flows is always a full template.
- No GitHub-style fallback defaults — an HMAC flow without a resolved
template cleanly 401s instead of silently behaving like a GitHub receiver.
- Safe-header logging replaced: pattern-based deny-list
(/secret|token|sig|hmac|signature|auth|password|bearer|api[-_]?key/i) plus
the hard blocks for authorization/cookie/proxy-authorization. No provider
names in the logging whitelist.
- CLI drops --signature-header / --signature-format / --auth-mode=hmac_sha256.
New surface: --preset=<name> (filter-registered) or --config=@template.json
(explicit) for HMAC; bearer otherwise. Core ships zero presets.
- The stored flow row never records a preset name — the preset is expanded
to a full template at enable time. Changing a preset registration after
enable doesn't silently mutate configured flows.
New CLI subcommands
- `rotate <id> --generate [--previous-ttl-seconds=N]` — zero-downtime
secret rotation. Demotes current → previous with a TTL, installs fresh
current. Both verify until previous expires.
- `forget <id> <secret_id>` — immediate removal from the rotation list.
- `presets` — list filter-registered presets (empty by default).
New abilities
- `datamachine/webhook-trigger-rotate-secret`
- `datamachine/webhook-trigger-forget-secret`
Backward compatibility
- Every shipped v1 flow keeps working. Migration happens silently on first
read via WebhookAuthResolver::migrate_legacy(): the v1 fields
(webhook_auth_mode=hmac_sha256, webhook_signature_header,
webhook_signature_format, webhook_secret) are converted to the canonical
v2 shape (webhook_auth_mode=hmac, webhook_auth, webhook_secrets) and the
legacy fields are deleted. The migration is covered by a dedicated
end-to-end test.
- Bearer flows are untouched.
- `WebhookSignatureVerifier` is marked @deprecated but kept to avoid
breaking unknown external callers.
Tests (+58 new, 0 regressions)
- WebhookVerifierTest: 29 tests. Pure unit. Provider-shaped matrix (prefixed
hex / base64 header / raw hex / composite kv+timestamp / separate timestamp
/ id+timestamp / url+params) + replay edges + rotation + expiry + error
paths. Matrix entries are shape-named, not provider-named.
- WebhookAuthResolverTest: 13 tests. v1→v2 migration (happy path + orphan
fields + no-op cases), preset filter lookup, deep-merge.
- WebhookTriggerTest (rewritten): 17 tests. Bearer regression, HMAC via
preset + via explicit template + with overrides, template deep-merge,
no-template-set returns 401 (not GitHub default), rotation grace window,
forget invalidation, silent v1 migration, safe-headers deny-list pattern.
- WebhookSignatureVerifierTest preserved for deprecated-shim regression.
Docs
- docs/api/endpoints/webhook-triggers.md rewritten without a provider
table. Describes the template grammar, extract kinds, encodings, preset
filter, rotation workflow, and backward-compat migration. Lists provider
names only in the non-HMAC modes section (Ed25519 for Discord, x509 for
AWS SNS) — those are filter extension-point examples, not core features.
- docs/core-system/wp-cli.md updated with the new CLI surface.
- docs/core-system/abilities-api.md: webhook section is now 8 abilities.
Explicitly out of scope (future follow-ups)
- Dedicated `{prefix}_webhook_secrets` table with migration.
- Structured verification log table.
- Outbound (AgentPing) signing via the same grammar.
- Nonce-based replay storage beyond timestamp window.
- Concrete Ed25519 / x509 / JWT implementations for the mode registry.
Refs: #1179
Contributor
Homeboy Results —
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1179. Replaces the (closed, unmerged) #1186 rebuild.
Why this exists
#1186 shipped the template engine but leaked provider-specific knowledge back into DM core in three places:
'sha256=hex' | 'hex' | 'base64'switch in the resolver.X-Hub-Signature-256/sha256=hex) used as silent fallbacks in the ability.stripe-signature,x-slack-signature,x-shopify-hmac-sha256, etc.).The constraint was strict: no provider names anywhere in core, everything supported via flow config. #1186 violated that constraint. This PR is the rebuild that actually honors it.
What's different from #1186
auth_mode = hmac_sha256anywhere. Replaced with the generic primitivehmac.--signature-header,--signature-format,--auth-mode=hmac_sha256, and the v1 fields they wrote (webhook_signature_header,webhook_signature_format,webhook_secret) are gone from the user-facing surface. Flows enable via--preset=<name>(filter-registered) or--config=@template.json(explicit template)./secret|token|sig|hmac|signature|auth|password|bearer|api[-_]?key/i+ hard blocks forauthorization/cookie/proxy-authorization). Zero provider names in the logging code.WebhookAuthResolver::migrate_legacy(), called once on first read, which converts and then deletes the v1 fields. No other code path references legacy field names.The architecture
scheduling_config = [ 'webhook_enabled' => true, 'webhook_auth_mode' => 'bearer' | 'hmac', // bearer-only: 'webhook_token' => '...', // hmac-only — always a full template when present: 'webhook_auth' => [ 'mode' => 'hmac', 'algo' => 'sha256', 'signed_template' => '{timestamp}.{body}', // {body} {timestamp} {id} {url} {header:X} {param:X} 'signature_source' => [ header|param, extract, encoding ], 'timestamp_source' => [ header|param, extract, format ], // optional → enables replay protection 'id_source' => [ header|param, extract ], // optional 'tolerance_seconds'=> 300, 'max_body_bytes' => 1048576, ], 'webhook_secrets' => [ ['id' => 'current', 'value' => '...'], ... ], ]Extract kinds:
raw|prefix|kv_pairs|regex. Encodings:hex|base64|base64url. Timestamp formats:unix|unix_ms|iso8601.Extension points
datamachine_webhook_auth_presets— third parties register provider shorthands. Core ships zero presets.datamachine_webhook_verifier_modes— third parties register non-HMAC modes (Ed25519, x509, JWT, mTLS). Core ships onlyhmac.New CLI surface
Tests
+58 new tests; zero new regressions.
WebhookVerifierTest(29) — pure unit. Shape-named matrix (prefixed_hex / base64_header / raw_hex / kv_timestamped / separate_timestamp / id_timestamped / url_params) × (valid / tampered / wrong secret) + replay edges + multi-secret rotation + expiry + all error paths.WebhookAuthResolverTest(13) — v1→v2 migration including orphan-field cleanup + no-op cases + preset filter + deep-merge.WebhookTriggerTest(17) — end-to-end. Bearer regression, HMAC via preset + explicit template + overrides, no-template-returns-401 (not GitHub default), rotation grace window, forget invalidation, silent v1 migration through the REST handler, pattern-based safe-headers deny-list.WebhookSignatureVerifierTest(15) — kept as regression coverage for the deprecated v1 shim.Lint
Clean for new/modified files. The only phpstan finding attributable to
WebhookTriggerAbilityis a sharedFlowHelperstrait signal that predates this PR.Backward compatibility — verified by test
test_v1_legacy_flow_migrates_silently_and_still_authenticates:Bearer flows are untouched.
LOC
2,976 insertions, 779 deletions = net +2,197 LOC including tests and docs. The engine + resolver + result is ~790 LOC; the rest is abilities / CLI / tests / docs.
Out of scope (future issues)
{prefix}_webhook_secretstable with migration.