diff --git a/.changeset/4278-get-media-buys-webhook-activity.md b/.changeset/4278-get-media-buys-webhook-activity.md new file mode 100644 index 0000000000..404a0ff1b2 --- /dev/null +++ b/.changeset/4278-get-media-buys-webhook-activity.md @@ -0,0 +1,62 @@ +--- +"adcontextprotocol": minor +--- + +Buyer-side webhook delivery visibility for AdCP 3.1, landing #4278 alongside #4582 track 4 (standardized log surface). Two new request fields, one new response field, two new shared core schemas, and the canonical pattern documentation that future resources will follow. + +### Request additions (`get-media-buys-request.json`) + +- `include_webhook_activity` (boolean, default `false`) — when true, each returned media buy MAY include a `webhook_activity` array describing recent reporting and health webhook fires for the calling principal. +- `webhook_activity_limit` (integer, 1–200, default 50) — per-buy cap on returned records, most-recent first. + +The two request-field names are now the **canonical opt-in convention** for any AdCP resource exposing `webhook_activity[]` (see snapshot-and-log.mdx § Webhook activity log pattern). + +### Response addition (`get-media-buys-response.json#/properties/media_buys/items`) + +- `webhook_activity[]` — `$ref`s the new canonical record at `/schemas/core/webhook-activity-record.json`. + +### New shared core schemas (#4582 track 4) + +- **`/schemas/core/webhook-activity-record.json`** — canonical record shape for a single webhook delivery attempt, intended to be `$ref`'d from any resource read that surfaces a `webhook_activity[]` log. Fields: `idempotency_key` (equals the payload's dedup key — no parallel `delivery_id`), `subscriber_id` (reserved for multi-subscriber configurations; precedent #3009), `fired_at`, `completed_at`, `notification_type` (refs the shared notification-type enum; adopters MUST add their types to that registry rather than minting a parallel enum), `sequence_number`, `attempt` (one record per attempt), `status` (`success` / `failed` / `timeout` / `connection_error` / `pending`), `url` (query+fragment stripped, secret-shaped path segments SHOULD be redacted), `http_status_code`, `response_time_ms`, `payload_size_bytes`, `error_message` (server-side classification only — never bodies or headers), and `ext` (resource-specific extension envelope per the standard AdCP pattern). Nullable fields use the draft-07 union-type idiom (`"type": ["string", "null"]` etc.); the spec's `nullable: true` OpenAPI shorthand is not part of draft-07 and is not used. Top-level `additionalProperties: false` — resource-specific extensions go on `ext`, not as ad-hoc top-level fields. This is a **deliberate departure** from the surrounding convention (every other core schema with an `ext` slot uses `additionalProperties: true`) and is the structural enforcement of the "uniform across resources" promise that justifies the hoist; future schema reviewers should not "fix" it back to `true`. +- **`/schemas/core/truncation-sentinel.json`** — universal AdCP sentinel for fields whose content has been truncated due to a size cap. Shape: `{ "_truncation": { "original_size_bytes": N, "preview": "...", "preview_format": "" } }`. The leading-underscore `_truncation` key is the discriminator — receivers detect a sentinel by testing `'_truncation' in value`, no redundant boolean. `_truncation.additionalProperties: true` so future revisions can add classification fields without a forward-compat break. `preview_format` is an open string with `text` / `json` / `base64` / `xml` / `html` listed as common values; receivers SHOULD treat unknown values as `text`. The description carries the canonical `oneOf` usage example so the first real consumer doesn't reinvent the discriminator convention. Lands now so future RFCs (notably the `include_webhook_payloads` extension) plug into a shared shape; no field uses it today. + +### Normative rules (#4582 track 4) + +- **Retention is MUST, not SHOULD.** Sellers that surface `webhook_activity[]` MUST retain records for at least 30 days from each record's `completed_at`. For records still in `pending` status the clock runs from `fired_at` until the attempt terminates and then resets to 30 days from `completed_at` — so retry trails do not age out mid-flight. Sellers that cannot honor the floor MUST omit the field entirely rather than return a shorter window. This gives buyers a single retention guarantee they can build debug tooling against, and gives sellers with thin storage a clean opt-out via the three-state presence semantics rather than per-seller-negotiated floors. Resolves #4278 open question. +- **Scoping** MUST be calling-principal only even when multiple principals share visibility into the same resource via account-level access. +- **One record per attempt.** Single-attempt successes appear as a single record with `attempt: 1`; retry trails appear as multiple records sharing `idempotency_key`. +- **Three-state presence.** Field omitted = seller does not surface (no persistence, OR capability surface excludes the relevant webhook channel, OR no registered endpoint for the principal); `[]` = persists but no recent fires; non-empty = actual records. Sellers MUST NOT collapse states. +- **URL privacy.** Query string and fragment MUST be stripped. Sellers SHOULD redact path segments matching obvious secret patterns (high-entropy random material, UUID / token shapes). +- **`error_message` privacy.** Server-side classification string only — never request headers, response bodies, or buyer-endpoint stack traces. + +### Documentation + +- New normative section **`docs/protocol/snapshot-and-log.mdx` § Webhook activity log pattern** — names the canonical record, the two request-field conventions, scoping, retention floor, three-state presence semantics, record cardinality, and privacy rules. Includes an explicit **8-item adoption checklist** so future resources have unambiguous MUST hooks. Item 1 is the **notification-channel prerequisite**: adoption requires a registered notification channel for the relevant fire types — per-buy `push_notification_config` (existing) for buy-scoped resources, or the per-account subscription model from #4582 track 3 for resources that outlive a buy. The two are different primitives that fulfill the same prerequisite. Without a channel there are no fires to log, so the rest of the checklist is gated on this item. The earlier media-buy-specific mention now cross-references the pattern. Buyers diagnosing an unexpected omission have two observable signals (`push_notification_config` registration state, seller capability declaration) to discriminate the cause without filing a ticket. +- New "Diagnosing missing fires" subsection in `docs/building/by-layer/L3/webhooks.mdx` so buyers triaging missing fires from the transport contract page can find the debug surface. +- `docs/media-buy/task-reference/get_media_buys.mdx` documents `include_webhook_activity` / `webhook_activity_limit` / `webhook_activity[]` with field table, status semantics, three-state presence, retention MUST, and a JS+Python "diagnose a webhook delivery problem" example that groups attempts by `idempotency_key` and selects the latest attempt by `attempt` number (robust against iteration order). + +### Scope of this PR within #4582 + +- **Track 1** (snapshot/log duality doc) — already shipped at `docs/protocol/snapshot-and-log.mdx`; this PR extends it with the Webhook activity log pattern section. +- **Track 2** (persistent webhook contract) — already shipped at `docs/building/by-layer/L3/webhooks.mdx`; this PR adds the cross-link from the contract page back into the debug surface. +- **Track 3** (per-account subscription model) — explicitly **not** in this PR; targeted for 3.2.0 because it introduces a new account-level surface that needs to compose carefully with #3009 (multi-subscriber, 4.0). +- **Track 4** (standardized log surface) — **shipped here**: hoisted record schema, universal truncation sentinel, retention MUST resolution, canonical pattern documentation. +- **Tracks 5–7** (auth/transport hygiene, dedup edge cases, conformance rendezvous) — separate cadence per the epic. + +### Dependency chain (informational) + +Track 4's adoption checklist names a notification-channel prerequisite as item 1. The implication: media buys adopt today because their channel (per-buy `push_notification_config`) already exists. Resources that outlive a media buy — creative-lifecycle (#2261), audiences, properties, account-level governance (#1711) — are blocked on track 3 (3.2.0) for the per-account channel. Once track 3 ships, those consumers plug into this pattern's record shape, request fields, scoping, retention floor, and three-state presence — inheriting transport, subscription, and observability from #4582 rather than re-deriving any of them. The #2261 RFC itself scopes to creative-specific event payloads + state-machine transitions; everything else is inherited. + +### Backwards compatibility + +Both request fields are optional with default `false` / `50`; the response field is optional and absent unless `include_webhook_activity: true` is set AND the seller surfaces fire history for the buy with the required retention floor. Old clients see no change. + +### Out of scope (future work) + +- **`include_webhook_payloads`** — sensitive opt-in to surface request and response bodies. Carved out as a separate extension because request/response bodies warrant stricter access controls and would consume the new truncation sentinel for size-bounding. +- **Operator-facing aggregate views** across principals. +- **Cross-subscriber visibility** under #3009 — `subscriber_id` is reserved on the record shape now so #3009 can populate it without a schema break. +- **Real-time push** of webhook-activity events. +- **Replay tool** (re-fire a past delivery). + +Closes #4278. Lands #4582 track 4. diff --git a/docs/building/by-layer/L3/webhooks.mdx b/docs/building/by-layer/L3/webhooks.mdx index 2c7c4b1b1c..5fa7addb6c 100644 --- a/docs/building/by-layer/L3/webhooks.mdx +++ b/docs/building/by-layer/L3/webhooks.mdx @@ -444,6 +444,10 @@ app.post('/webhooks/adcp', async (req, res) => { **Always implement polling as backup.** Webhooks can fail due to network issues or server downtime. Use a slower poll interval when webhooks are configured (e.g., every 2 minutes instead of 30 seconds), and stop polling once you receive a terminal status via webhook. +### Diagnosing missing fires + +When a buyer suspects a webhook isn't reaching its endpoint — gateway 5xx, stale-sequence dedup, drifted webhook URL, suppressed fires under a tripped circuit breaker — call [`get_media_buys`](/docs/media-buy/task-reference/get_media_buys#webhook-activity) with `include_webhook_activity: true`. Each returned media buy will carry a `webhook_activity` array of recent fires for the calling principal, including `idempotency_key` (matches the payload's dedup key — correlate against your own endpoint log), `status` (`success` / `failed` / `timeout` / `connection_error` / `pending`), `http_status_code`, `attempt`, and `error_message`. The scope is the calling principal's own fires; no operator ticket required. + ## Best practices 1. **Always implement polling as backup** — webhooks can fail; poll at a reduced interval (e.g. every 2 minutes) when webhooks are configured, and stop once you receive a terminal status diff --git a/docs/media-buy/task-reference/get_media_buys.mdx b/docs/media-buy/task-reference/get_media_buys.mdx index 8e5e0038eb..bd07cd5d2a 100644 --- a/docs/media-buy/task-reference/get_media_buys.mdx +++ b/docs/media-buy/task-reference/get_media_buys.mdx @@ -30,6 +30,8 @@ Sellers that need to partition inventory away from a caller MUST do so at the ** | `status_filter` | string \| string[] | No | Status filter: `"pending_creatives"`, `"pending_start"`, `"active"`, `"paused"`, `"completed"`, `"rejected"`, `"canceled"`. Defaults to `["active"]` only when `media_buy_ids` is omitted. | | `include_snapshot` | boolean | No | When true, include near-real-time delivery snapshots for each package. Defaults to `false`. | | `include_history` | integer | No | Include the last N revision history entries per media buy (returns min(N, available)). 0 or omit to exclude. Max 1000. | +| `include_webhook_activity` | boolean | No | When true, each media buy includes a `webhook_activity` array with recent delivery-report webhook fires for the calling principal. Defaults to `false`. See [Webhook activity](#webhook-activity). | +| `webhook_activity_limit` | integer | No | Per-buy cap on returned webhook records (most-recent first). Range 1–200, default 50. Ignored when `include_webhook_activity` is false. | | `pagination` | object | No | Cursor-based pagination controls (`max_results`, `cursor`) for broad queries. | *`media_buy_ids` filters results to specific media buys. If neither is provided, the query is scope-based and uses `status_filter` + `pagination`. @@ -61,6 +63,7 @@ Returns an array of media buys with current status, creative approval state, and | `revision` | Current revision number. Pass in `update_media_buy` for optimistic concurrency. | | `valid_actions` | Actions the buyer can perform in the current state (e.g., `["pause", "cancel", "update_budget"]`). See [valid actions mapping](#valid-actions-mapping). | | `history` | Revision history entries, most recent first. Only present when `include_history > 0`. Append-only — entries are never modified or deleted. | +| `webhook_activity` | Recent reporting and health webhook fires for the calling principal, most-recent first. Only present when `include_webhook_activity` is true AND the seller surfaces fire history for this buy. See [Webhook activity](#webhook-activity) for three-state presence semantics. | | `packages` | Array of packages with creative status and optional snapshots | ### Package Object @@ -122,6 +125,180 @@ History entries are **append-only** — sellers MUST NOT modify or delete previo Money fields use this currency precedence: `snapshot.currency` -> `package.currency` -> `media_buy.currency`. +### Webhook Activity + +When `include_webhook_activity: true`, each returned media buy MAY carry a `webhook_activity` array describing recent reporting and health webhook fires from the seller to the buyer's registered endpoint. This is the buyer-side debug surface for the [persistent-channel webhook contract](/docs/building/by-layer/L3/webhooks#persistent-channel-contract) — buyers use it to verify that the publisher fired, what the buyer's gateway returned, and whether retries are still in flight, without needing an operator-level query against the seller's logs. + +The record shape, request-field names, scoping, retention floor, three-state presence, and cardinality rules are uniform across AdCP resources that adopt this surface. See [Webhook activity log pattern](/docs/protocol/snapshot-and-log#webhook-activity-log-pattern) on the snapshot/log contract page for the cross-resource normative section — the rules below restate it for the media-buy call site and add the media-buy-specific capability gate. + +The surface covers both delivery-report notification types (`scheduled`, `final`, `delayed`, `adjusted`) and health notification types (`impairment`). All share the same webhook delivery contract and the same buyer-side debug need. + +| Field | Description | +|-------|-------------| +| `idempotency_key` | Equals the `idempotency_key` carried in the webhook payload itself ([§ Dedup by `idempotency_key`](/docs/building/by-layer/L3/webhooks#dedup-by-idempotency_key)). Stable across retries of the same logical fire — buyers correlate this surface with their own endpoint logs via this exact field. Reference it when filing support tickets. | +| `subscriber_id` | Identifies which registered webhook subscriber received this fire. Absent in single-subscriber configurations; populated under multi-subscriber buys (4.0+, see [#3009](https://github.com/adcontextprotocol/adcp/issues/3009)). | +| `fired_at` | ISO 8601 timestamp when the seller initiated this attempt. | +| `completed_at` | ISO 8601 timestamp when the response was observed (or terminal failure). Null while `status` is `pending`. | +| `notification_type` | Verbatim from the webhook payload: `scheduled`, `final`, `delayed`, `adjusted`, `impairment`. | +| `sequence_number` | Sequence number from the webhook payload — useful for spotting stale-sequence drops or gaps. Absent when the notification type does not carry one. | +| `attempt` | 1-indexed retry counter for this logical fire. Initial fire is `attempt: 1`. | +| `status` | `success`, `failed`, `timeout`, `connection_error`, or `pending`. See semantics below. | +| `url` | Target URL with **query string and fragment stripped**, and high-entropy / token-shaped path segments redacted. Match this against your registered URL by origin + path, not full URL. | +| `http_status_code` | HTTP status from the buyer's endpoint. Null when no HTTP response was received (`timeout`, `connection_error`, `pending`). | +| `response_time_ms` | Wall-clock latency between request send and response receipt. Null for non-completed attempts. | +| `payload_size_bytes` | Size of the request body the seller sent — useful for diagnosing oversized-payload rejections. | +| `error_message` | Short human-readable server-side classification of failure. Null for `success`. Sellers MUST NOT include request / response bodies or headers here. | + +**Status semantics:** + +- `success` — response received with a 2xx status. `http_status_code` populated. +- `failed` — response received with a non-2xx status. `http_status_code` populated; `error_message` describes the response. +- `timeout` — no response within the seller's configured timeout. `http_status_code` null. Operationally: the buyer's endpoint is reachable but slow / overloaded. +- `connection_error` — DNS, TLS, or socket failure before any HTTP response. `http_status_code` null. Operationally: the buyer's endpoint is unreachable or misconfigured. +- `pending` — attempt is in flight or queued for retry. `completed_at` is null; subsequent attempts appear with the same `idempotency_key` and incremented `attempt`. + +**Record cardinality:** one record per attempt. A successful first-attempt fire appears as a single record with `attempt: 1`. A 3-attempt retry trail (e.g., two failures then a success) appears as three records sharing `idempotency_key`. + +**Scoping (normative):** + +- `webhook_activity` MUST be scoped to the **calling principal**. When multiple buyer principals share visibility into the same media buy via account-level access, each principal sees only fires targeting its own endpoint. +- Sellers that surface this field **MUST** retain records for at least 30 days from each record's `completed_at` — uniformly across `success`, `failed`, `timeout`, and `connection_error` outcomes (all of which populate `completed_at`). For records still in `pending` status, the clock runs from `fired_at` until the attempt terminates and then transitions to 30 days from `completed_at` — retry trails do not age out mid-flight. Sellers that cannot honor this floor MUST omit the field entirely rather than return a shorter window; the three-state presence semantics give them a clean opt-out and buyers a single guarantee they can build against. +- This surface is a debug aid, not a full audit log. There is no cursor for older fires beyond `webhook_activity_limit` — buyers needing full history must persist webhook records on their own side. + +**Three-state presence semantics:** + +| State | Meaning | +|-------|---------| +| Field **omitted** | Seller does not surface webhook activity for this buy. Either the seller does not persist fire history, the seller's declared [`propagation_surfaces`](/docs/media-buy/media-buys/lifecycle) excludes `webhook`, or the buy has no registered webhook endpoint for the calling principal. | +| Empty array `[]` | Seller persists fire history but has fired nothing recent for this principal. | +| Non-empty array | Actual fire records, most-recent first. | + +Sellers whose declared `propagation_surfaces` does not include `webhook` MUST omit the field; opting in via `include_webhook_activity: true` does not override that. + +**Diagnosing an unexpected omission.** When you expected fires but got an omitted field, two observables let you discriminate the cause without filing a ticket: (1) check your own `push_notification_config` registration state for this buy — if not registered, that's the cause; (2) check the seller's `capabilities.media_buy.propagation_surfaces` via `get_adcp_capabilities` — if `webhook` is absent, that's the cause. When both check out, "seller does not persist fire history" is the remaining cause; that's a seller-side gap and warrants an operator ticket. + +**Privacy:** + +- The `url` field has its query string and fragment **stripped**, and sellers SHOULD redact path segments resembling shared secrets (high-entropy random material, UUID / token shapes). +- Request and response bodies are **not surfaced** by this field. A future `include_webhook_payloads` extension may add them under stricter authorization controls — out of scope here. +- `error_message` is a server-side classification string only — never request headers, never response bodies, never buyer-endpoint stack traces. + +#### Diagnose a webhook delivery problem + + + +```typescript TypeScript +import { testAgent } from '@adcp/sdk/testing'; +import { GetMediaBuysResponseSchema, type WebhookActivityRecord } from '@adcp/sdk'; + +// The WebhookActivityRecord type is regenerated by the SDK from +// /schemas/core/webhook-activity-record.json — once the SDK rebuilds against this +// branch's schemas the import resolves. The same type appears on every AdCP resource +// that surfaces webhook_activity[], so debug helpers can be written once and reused. +function latestAttempt(trail: WebhookActivityRecord[]): WebhookActivityRecord { + return trail.reduce((a, b) => (a.attempt >= b.attempt ? a : b)); +} + +const result = await testAgent.getMediaBuys({ + media_buy_ids: ['mb_12345'], + include_webhook_activity: true, + webhook_activity_limit: 20, +}); + +if (!result.success) { + throw new Error(`Request failed: ${result.error}`); +} + +const validated = GetMediaBuysResponseSchema.parse(result.data); + +for (const mediaBuy of validated.media_buys) { + // Three-state semantics — distinguish "seller does not surface" from "no recent fires". + if (mediaBuy.webhook_activity === undefined) { + console.log(`${mediaBuy.media_buy_id}: seller does not surface webhook activity for this buy`); + continue; + } + + const fires = mediaBuy.webhook_activity; + if (fires.length === 0) { + console.log(`${mediaBuy.media_buy_id}: no recent fires for this principal`); + continue; + } + + // Group attempts by idempotency_key so we can see the retry trail per logical fire. + const trails = new Map(); + for (const fire of fires) { + const trail = trails.get(fire.idempotency_key) ?? []; + trail.push(fire); + trails.set(fire.idempotency_key, trail); + } + + for (const [idempotencyKey, trail] of trails) { + // Pick the latest attempt by `attempt` number — robust against any iteration order. + const latest = latestAttempt(trail); + if (latest.status === 'success') continue; + + const detail = latest.error_message ?? latest.http_status_code ?? '—'; + console.log( + `${mediaBuy.media_buy_id} ${idempotencyKey} ` + + `(${latest.notification_type} seq=${latest.sequence_number}): ` + + `${latest.status} after ${trail.length} attempt(s) — ${detail}` + ); + } +} +``` + +```python Python +import asyncio +from collections import defaultdict +from adcp.testing import test_agent +from adcp.types import GetMediaBuysRequest, WebhookActivityRecord + +# WebhookActivityRecord is regenerated by the SDK from +# /schemas/core/webhook-activity-record.json — once the SDK rebuilds against this +# branch's schemas the import resolves. The same type appears on every AdCP resource +# that surfaces webhook_activity[]. +def latest_attempt(trail: list[WebhookActivityRecord]) -> WebhookActivityRecord: + return max(trail, key=lambda f: f.attempt) + +async def main(): + result = await test_agent.get_media_buys( + GetMediaBuysRequest( + media_buy_ids=['mb_12345'], + include_webhook_activity=True, + webhook_activity_limit=20, + ) + ) + + for media_buy in result.media_buys: + # Three-state semantics: distinguish "seller does not surface" from "no recent fires". + if media_buy.webhook_activity is None: + print(f"{media_buy.media_buy_id}: seller does not surface webhook activity for this buy") + continue + + fires = media_buy.webhook_activity + if not fires: + print(f"{media_buy.media_buy_id}: no recent fires for this principal") + continue + + trails = defaultdict(list) + for fire in fires: + trails[fire.idempotency_key].append(fire) + + for idempotency_key, trail in trails.items(): + latest = latest_attempt(trail) + if latest.status == 'success': + continue + + detail = latest.error_message or latest.http_status_code or '—' + print(f"{media_buy.media_buy_id} {idempotency_key} " + f"({latest.notification_type} seq={latest.sequence_number}): " + f"{latest.status} after {len(trail)} attempt(s) — {detail}") + +asyncio.run(main()) +``` + + + ## Valid Actions Mapping The `valid_actions` array tells agents what operations are permitted on a media buy in its current state. Sellers SHOULD include this field. Expected values by status: diff --git a/docs/protocol/snapshot-and-log.mdx b/docs/protocol/snapshot-and-log.mdx index 144ad6515c..a4e59b3b91 100644 --- a/docs/protocol/snapshot-and-log.mdx +++ b/docs/protocol/snapshot-and-log.mdx @@ -77,6 +77,84 @@ Without two-paths-equal, AdCP becomes pub/sub for some channels and REST for oth A webhook delivery surfaced via `webhook_activity[]` references the same `notification_id` that the buyer received in the push body. A buyer can correlate "I received fire X" with "the seller's log shows fire X" without bookkeeping across two namespaces. Likewise, an `impairment_id` referenced in `impairments[]` matches the `notification_id` of the push that announced it. +## Webhook activity log pattern + +The transport half of Rule 5. Any AdCP resource that exposes a snapshot read API and has webhook fires associated with it MAY also surface a `webhook_activity[]` array on that read API — recent per-fire transport records, scoped to the calling principal, useful for buyer-side debugging when a fire didn't land or a retry trail looks suspect. This section is the contract any resource adopting that surface MUST follow. + +### Canonical record shape + +The record shape is fixed at [`/schemas/core/webhook-activity-record.json`](https://adcontextprotocol.org/schemas/v3/core/webhook-activity-record.json). Read schemas adopting this surface MUST `$ref` the canonical record rather than inline it — the shape is intentionally uniform across resources so a buyer's debug tooling can consume `webhook_activity[]` from any read API without resource-specific parsing. + +Each record carries `idempotency_key` (equals the payload's `idempotency_key` per Rule 5 — no parallel `delivery_id`), `subscriber_id` (reserved for #3009 multi-subscriber), `fired_at`, `completed_at`, `notification_type`, `sequence_number`, `attempt` (1-indexed; one record per attempt), `status` (`success` / `failed` / `timeout` / `connection_error` / `pending`), `url` (query string and fragment stripped, secret-shaped path segments redacted), `http_status_code`, `response_time_ms`, `payload_size_bytes`, and `error_message` (server-side classification only — never request/response bodies or headers). + +### Request-field convention + +Read schemas that surface `webhook_activity[]` MUST use the same two request-field names so callers can opt in uniformly across resources: + +- **`include_webhook_activity`** — boolean, default `false`. When true, the seller MAY return a `webhook_activity[]` array on each item (subject to the three-state presence semantics below). +- **`webhook_activity_limit`** — integer, range 1–200, default 50. Per-item cap on returned records, most-recent first. + +### Scoping (normative) + +`webhook_activity[]` MUST be scoped to the **calling principal**. When multiple principals share visibility into the same resource via account-level access, each principal sees only fires targeting its own registered endpoint. This is the same scoping rule that applies to push delivery itself. + +### Retention (normative) + +Sellers that surface `webhook_activity[]` **MUST** retain records for at least 30 days from each record's `completed_at`. This applies uniformly to every terminal status — `success`, `failed`, `timeout`, and `connection_error` all populate `completed_at` (for `timeout` and `connection_error` it is the moment the seller declared the attempt terminal) and the 30-day clock runs from there. For records still in `pending` status (the attempt is in flight or queued for retry, `completed_at` is null), the clock runs from `fired_at` until the attempt terminates and then transitions to 30 days from `completed_at` — so a retry trail does not age out mid-flight just because the initial fire happened 29 days ago. + +The 30-day floor is a hard contract — sellers unable to honor it MUST omit the field entirely (see three-state presence below) rather than return a shorter window. This gives buyers a single retention guarantee they can build debug tooling against, and gives sellers with thin storage a clean opt-out via the three-state semantics rather than forcing the spec to negotiate per-seller retention floors. + +### Three-state presence semantics + +| State | Meaning | +|-------|---------| +| Field **omitted** | Seller does not surface webhook activity for this resource. Causes are resource-specific (see "Adoption checklist" below) but typically include: the seller does not persist fire history; the resource has no registered webhook endpoint for the calling principal; the seller's declared capability surface excludes the webhook channel for the relevant notification types. Buyers MUST NOT infer "no fires occurred" from omission. | +| Empty array `[]` | Seller persists fire history but has fired nothing recent for this principal. | +| Non-empty array | Actual fire records, most-recent first. | + +Sellers MUST NOT collapse these into a single state. Opting in via `include_webhook_activity: true` does not override the seller's intrinsic capability — a seller that cannot meet the retention floor returns omission regardless of the request. + +Buyers diagnosing an unexpected omission have two readily observable signals to discriminate the cause without needing operator help: (1) their own `push_notification_config` registration state for the resource (rules out "no registered endpoint") and (2) the seller's capability declaration (rules out "capability surface excludes the channel"). When both check out, "seller does not persist fire history" is the remaining cause and no further protocol-side fix is available — escalate. + +### Record cardinality + +One record per attempt. A successful first-attempt fire appears as a single record with `attempt: 1`. A 3-attempt retry trail (e.g., two failures then a success) appears as three records sharing `idempotency_key` — the trail is reconstructed by the buyer grouping records on that key. + +### Privacy + +- `url` MUST have query string and fragment stripped, and high-entropy / token-shaped path segments SHOULD be further redacted. +- `error_message` is a server-side classification string only — never request headers, response bodies, or buyer-endpoint stack traces. +- Request and response bodies are out of scope for the basic surface. A future `include_webhook_payloads` extension may add them under stricter access controls, and would use the [universal truncation sentinel](https://adcontextprotocol.org/schemas/v3/core/truncation-sentinel.json) at `/schemas/core/truncation-sentinel.json` when bodies exceed a configured cap. + +### Adoption checklist + +Resources adopting `webhook_activity[]` MUST satisfy all of the following. The list is intentionally explicit so the "MUST" hooks are unambiguous; everything not on this list is at adopter discretion (e.g., per-resource cardinality tuning within the 1–200 range). + +1. **Notification channel (prerequisite).** Adoption requires a registered notification channel for the relevant fire types. Media buys satisfy this today via per-buy `push_notification_config` (and the related `reporting_webhook`); resources that outlive any single buy — creatives, audiences, properties, account-level governance — wait on the **per-account subscription model defined in #4582 track 3** (forthcoming in 3.2.0). The two are different primitives that fulfill the same prerequisite: a buy-scoped config blob attached to the buy versus an account-scoped subscription resource. Without a channel there are no fires for `webhook_activity[]` to log; this item gates every other rule below. Adopters MUST cite the specific channel in their call-site documentation. +2. **Record shape.** Item schema MUST `$ref` `/schemas/core/webhook-activity-record.json`. Resource-specific cross-references (e.g., a parent-resource id when records are nested inside an account-level read) go on the canonical record's `ext` envelope, not as top-level record fields. +3. **Request fields.** The opt-in field names MUST be `include_webhook_activity` (boolean, default `false`) and `webhook_activity_limit` (integer, 1–200, default 50). The 200 ceiling is the canonical cap; adopters MAY narrow the maximum on a per-resource basis but MUST NOT exceed 200 or rename the fields. +4. **Scoping.** MUST be calling-principal only, per § Scoping above. +5. **Retention floor.** MUST honor the 30-day floor per § Retention above. The pivot (`completed_at`, with carve-out for `pending`) is the same across resources. +6. **Three-state presence cardinality.** Omitted / `[]` / non-empty are the three states; adopters MUST NOT collapse them. +7. **Capability gate.** Adopters MUST document which resource-specific capability declaration gates the field (for media buys this is `capabilities.media_buy.propagation_surfaces` including `webhook`). The specific *causes* of the "field omitted" state ARE resource-specific and adopters MUST enumerate them in their call-site documentation; the cardinality and the rule that omission is not "no fires occurred" are universal. +8. **Notification type registry.** Adopters whose webhook fires carry notification types not in [`/schemas/enums/notification-type.json`](https://adcontextprotocol.org/schemas/v3/enums/notification-type.json) MUST add those types to that shared enum rather than minting a parallel enum on the canonical record. The enum is the cross-resource registry. + +### Consumers and the dependency chain + +#### Today (3.1) + +- `get_media_buys.media_buys[].webhook_activity[]` — the first and currently only consumer of this pattern. The notification channel is the existing per-buy `push_notification_config`, so item 1 of the checklist is satisfied without any new primitive. Capability gate: `capabilities.media_buy.propagation_surfaces` MUST include `webhook` for the field to be surfaced on a buy. See [get_media_buys § Webhook activity](/docs/media-buy/task-reference/get_media_buys#webhook-activity) for the call-site documentation and the [persistent webhook contract](/docs/building/by-layer/L3/webhooks#persistent-channel-contract) for the transport-side rules this surface debugs against. + +#### Blocked on track 3 (3.2.0+) + +Resources that outlive a single media buy cannot adopt this pattern in 3.1 because item 1 is unsatisfied; they are blocked on the per-account notification channel: + +1. **#4582 track 3 (3.2.0)** introduces the per-account subscription model — the prerequisite notification channel for resources that outlive a buy. No new fields land on this read pattern as part of track 3 itself; track 3 supplies the *registration* primitive that unlocks the pattern for resources that need it. +2. **[#2261](https://github.com/adcontextprotocol/adcp/issues/2261) creative-lifecycle webhooks** are blocked on track 3. Once track 3 lands, #2261 plugs into this pattern's record shape and request-field conventions, scoping its capability gate to the creative-specific declaration. The #2261 RFC itself scopes to creative event payloads + state-machine transitions; transport, subscription, and observability are inherited from #4582. +3. Other resources that outlive a buy (audiences, properties, account-level governance under [#1711](https://github.com/adcontextprotocol/adcp/issues/1711)) follow the same chain: track 3 first, then adoption. + +Adopters of either future variant follow this checklist verbatim once item 1 is satisfied. + ## What this rules out - **A push channel for suggestions that don't change state.** If "the seller wants you to know X" doesn't correspond to a readable field, it's not a snapshot/log event. Build a pull tool instead. (See the advisory epic.) @@ -88,7 +166,7 @@ A webhook delivery surfaced via `webhook_activity[]` references the same `notifi - **Delivery reports** (`scheduled` / `final` / `delayed` / `adjusted`) predate this contract. Rule 4 closes for them in 3.1 via two surfaces: - **Per-window data parity** — `get_media_buy_delivery` accepts `time_granularity` + `include_window_breakdown: true`, returning `media_buy_deliveries[].windows[]` slices shape-aligned with `reporting_webhook` payloads at the same granularity. Capability-scoped via `reporting_capabilities.windowed_pull_granularities`; pulls outside the declared set return `UNSUPPORTED_GRANULARITY`. Landed in #4590. - - **Per-fire transport log** — even with per-window parity, buyers debugging webhook delivery want to see which fires hit their endpoint and when. The `webhook_activity[]` surface on `get_media_buys` ([#4278](https://github.com/adcontextprotocol/adcp/issues/4278)) closes this for transport-layer observability. + - **Per-fire transport log** — even with per-window parity, buyers debugging webhook delivery want to see which fires hit their endpoint and when. The `webhook_activity[]` surface on `get_media_buys` ([#4278](https://github.com/adcontextprotocol/adcp/issues/4278)) closes this for transport-layer observability. It is the first consumer of the [webhook activity log pattern](#webhook-activity-log-pattern) above; future resources adopting the pattern follow the same record shape, retention floor, and three-state presence semantics. - **Resource lifecycle webhooks beyond media-buy scope** (creative state changes, audience suspensions outside a buy's scope) are in progress under the creative-lifecycle webhooks RFC ([#2261](https://github.com/adcontextprotocol/adcp/issues/2261)). Until those land, the snapshot half (a fresh `list_creatives` or `sync_audiences` call) is the only reliable signal for changes to resources not currently referenced by an active buy. ## When you'd be right to push back diff --git a/static/schemas/source/core/truncation-sentinel.json b/static/schemas/source/core/truncation-sentinel.json new file mode 100644 index 0000000000..4ba7a3b204 --- /dev/null +++ b/static/schemas/source/core/truncation-sentinel.json @@ -0,0 +1,36 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "/schemas/core/truncation-sentinel.json", + "title": "Truncation Sentinel", + "description": "Universal AdCP sentinel value that replaces a field's content when the seller has truncated it due to a size cap. Used wherever an AdCP field may carry large user-supplied content (e.g., webhook request/response payloads under a future `include_webhook_payloads` extension, large structured-event blobs, captured response bodies) and the seller has elected to surface a bounded preview instead of the full value. The leading-underscore key `_truncation` is a deliberate control marker — its presence on an object is the discriminator that distinguishes a truncation sentinel from the natural value the field would otherwise carry. Schemas that may carry truncated content reference this shape via `oneOf` against the natural value type — e.g., `{ \"oneOf\": [ { \"$ref\": \"#/definitions/NaturalValue\" }, { \"$ref\": \"/schemas/core/truncation-sentinel.json\" } ] }`. Receivers detect a sentinel by testing `'_truncation' in value`. See snapshot-and-log.mdx § Webhook activity log pattern for the cross-resource context this sentinel was introduced under.", + "type": "object", + "properties": { + "_truncation": { + "type": "object", + "description": "Truncation envelope. The leading underscore is a deliberate signal that this property is a control marker and not a payload field — the natural value being surfaced will not have a `_truncation` key. `additionalProperties: true` so future revisions of this contract can add classification fields without a forward-compat break.", + "properties": { + "original_size_bytes": { + "type": "integer", + "description": "Size of the untruncated original content in bytes, measured before any encoding overhead. Lets the receiver decide whether to fetch the full value via a payload-bearing surface (when one exists) or proceed with the preview.", + "minimum": 0 + }, + "preview": { + "type": "string", + "description": "Optional bounded human-readable excerpt of the original content. Sellers SHOULD keep previews under 8 KiB and SHOULD truncate on a UTF-8 codepoint boundary. Absent when the seller cannot safely surface even a preview (e.g., the original content is binary and not text-decodable, or seller policy forbids any payload surfacing). Receivers MUST NOT parse `preview` as a complete representation of the original value." + }, + "preview_format": { + "type": "string", + "description": "Hint for how to render `preview`. Common values: `text` (plain UTF-8), `json` (JSON fragment), `base64` (base64-encoded binary), `xml`, `html`. The list is open — adopters MAY use other MIME-derived shorthands. Receivers SHOULD treat unknown values as `text` and SHOULD NOT reject the sentinel when the value is unfamiliar." + } + }, + "required": [ + "original_size_bytes" + ], + "additionalProperties": true + } + }, + "required": [ + "_truncation" + ], + "additionalProperties": false +} diff --git a/static/schemas/source/core/webhook-activity-record.json b/static/schemas/source/core/webhook-activity-record.json new file mode 100644 index 0000000000..4bceee20d8 --- /dev/null +++ b/static/schemas/source/core/webhook-activity-record.json @@ -0,0 +1,91 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "/schemas/core/webhook-activity-record.json", + "title": "Webhook Activity Record", + "description": "Single webhook delivery attempt surfaced to a calling principal as a buyer-side debug aid. Represents one HTTP attempt of a logical webhook fire from the seller to the buyer's registered endpoint — retries of the same logical fire share `idempotency_key` and differ by `attempt`. This is the canonical record shape for any AdCP resource that exposes a `webhook_activity[]` log on its read API; see snapshot-and-log.mdx § Webhook activity log pattern for the full normative contract (scoping, retention, three-state presence, request-field conventions).", + "type": "object", + "properties": { + "idempotency_key": { + "type": "string", + "description": "Equals the `idempotency_key` carried in the webhook payload itself (see docs/building/by-layer/L3/webhooks.mdx § Dedup by `idempotency_key`). Stable across retry attempts of the same logical fire — retries with `attempt` > 1 reuse this key. Buyers correlate this surface with their own endpoint logs via this exact field; the spec deliberately reuses the payload key rather than minting a parallel `delivery_id` so callers do not need a join table. Format is sender-defined; callers MUST treat as opaque." + }, + "subscriber_id": { + "type": "string", + "description": "Identifies which registered webhook subscriber received this fire. Optional and absent in single-subscriber configurations (the calling principal is unambiguous). Reserved for multi-subscriber configurations where a single resource has more than one registered subscriber endpoint — the seller MUST populate this field on records returned to the calling principal so callers can attribute fires across endpoints. Buyers MUST NOT use absence as a signal that no other subscribers exist; that information is not exposed by this surface. Precedent: #3009 (multi-subscriber `reporting_webhook` on media buys in AdCP 4.0); future per-account notification reads follow the same shape." + }, + "fired_at": { + "type": "string", + "format": "date-time", + "description": "ISO 8601 timestamp when the seller initiated the HTTP request for this attempt." + }, + "completed_at": { + "type": ["string", "null"], + "format": "date-time", + "description": "ISO 8601 timestamp when the seller observed the response (or terminal timeout / connection error — for `timeout` and `connection_error` outcomes, `completed_at` is set to the moment the seller declared the attempt terminal). Explicitly `null` when the attempt is still in flight or queued for retry (status `pending`); MUST be set as `null` rather than omitted so callers can distinguish 'still in flight' from 'field missing'." + }, + "notification_type": { + "$ref": "/schemas/enums/notification-type.json", + "description": "Notification type carried by this fire, verbatim from the webhook payload. Includes both delivery-report types (`scheduled`, `final`, `delayed`, `adjusted`) and health-notification types (`impairment`). All share the same persistent-channel webhook contract and the same buyer-side debug need." + }, + "sequence_number": { + "type": "integer", + "description": "Sequence number from the webhook payload. Surfaced here so the buyer can spot stale-sequence drops and gaps without correlating against their own endpoint log. Absent for notification types that do not carry a sequence number.", + "minimum": 0 + }, + "attempt": { + "type": "integer", + "description": "1-indexed retry counter for this logical fire. Initial fire is attempt=1; retries increment. Sellers MUST emit one record per attempt, so a successful first-attempt fire appears as a single record with `attempt: 1` and a 3-attempt retry trail appears as three records sharing `idempotency_key`.", + "minimum": 1 + }, + "status": { + "type": "string", + "description": "Outcome of this attempt. `success` — response received with 2xx (`http_status_code` populated). `failed` — response received with non-2xx (`http_status_code` populated). `timeout` — no response within the seller's configured timeout (`http_status_code` null). `connection_error` — DNS / TLS / socket failure before any HTTP response (`http_status_code` null). `pending` — attempt is in flight or queued for retry (`completed_at` null, `http_status_code` null). The `timeout` / `connection_error` split is intentional and operationally distinct: `timeout` typically signals a slow / overloaded buyer endpoint, `connection_error` typically signals it is unreachable or misconfigured.", + "enum": [ + "success", + "failed", + "timeout", + "connection_error", + "pending" + ] + }, + "url": { + "type": "string", + "format": "uri", + "description": "Target URL for this fire. Query string and fragment MUST be stripped before surfacing — buyers commonly stash bearer tokens in the query string and sellers MUST NOT echo those back through this debug surface. Sellers SHOULD additionally redact path segments matching obvious secret patterns (e.g., a path segment that is high-entropy random material or matches a UUID / token format). Buyers matching this against their own configured URL should compare by origin + path; query strings will not match and that mismatch is expected." + }, + "http_status_code": { + "type": ["integer", "null"], + "description": "HTTP status code returned by the buyer's endpoint. Explicitly `null` when no HTTP response was received (status `timeout`, `connection_error`, or `pending`); MUST be set as `null` rather than omitted.", + "minimum": 100, + "maximum": 599 + }, + "response_time_ms": { + "type": ["integer", "null"], + "description": "Wall-clock latency between request send and response receipt, in milliseconds. Explicitly `null` when the attempt did not complete (`timeout`, `connection_error`, `pending`); MUST be set as `null` rather than omitted.", + "minimum": 0 + }, + "payload_size_bytes": { + "type": "integer", + "description": "Size of the request body the seller sent, in bytes. Useful for diagnosing oversized-payload rejections from the buyer's gateway.", + "minimum": 0 + }, + "error_message": { + "type": ["string", "null"], + "description": "Short human-readable server-side classification of why this attempt did not succeed (e.g., `connection refused`, `TLS handshake timeout`, `HTTP 503 Service Unavailable`). Explicitly `null` for `success` (MUST be set as `null` rather than omitted). Sellers MUST NOT include request headers, request body content, or response body content in this field — payload surfacing is reserved for a future `include_webhook_payloads` extension and is subject to stricter access controls. Sellers SHOULD also avoid including buyer-endpoint internal hostnames, stack traces, or other implementation detail leaked by the response — keep it a stable classification string.", + "maxLength": 500 + }, + "ext": { + "$ref": "/schemas/core/ext.json", + "description": "Resource-specific extension slot. Adopters MAY surface a resource-specific cross-reference (e.g., `creative_id` on a creative-lifecycle record, `media_buy_id` on a record nested inside an account-level read) under `ext` rather than adding top-level fields — the canonical record shape stays uniform across resources and the `ext` envelope absorbs per-resource needs. Top-level extensions are not permitted (`additionalProperties: false`)." + } + }, + "required": [ + "idempotency_key", + "fired_at", + "notification_type", + "attempt", + "status", + "url" + ], + "additionalProperties": false +} diff --git a/static/schemas/source/media-buy/get-media-buys-request.json b/static/schemas/source/media-buy/get-media-buys-request.json index 78cc7f466f..d2b2f8f520 100644 --- a/static/schemas/source/media-buy/get-media-buys-request.json +++ b/static/schemas/source/media-buy/get-media-buys-request.json @@ -50,6 +50,18 @@ "maximum": 1000, "default": 0 }, + "include_webhook_activity": { + "type": "boolean", + "description": "When true, each returned media buy includes a `webhook_activity` array describing recent delivery-report webhook fires for the calling principal. Used by buyer agents to verify whether a publisher actually fired against the buyer's registered endpoint and what the endpoint returned — closes the operator-ticket loop for webhook debugging. Scoped to the calling principal: a buyer sees only fires targeting its own endpoint, even when multiple principals share visibility into the same media buy. Defaults to false. See `webhook_activity_limit` for the per-buy cap.", + "default": false + }, + "webhook_activity_limit": { + "type": "integer", + "description": "Maximum number of webhook delivery records to return per media buy, ordered most-recent first. Ignored when `include_webhook_activity` is false. Sellers that surface webhook activity MUST retain records for at least 30 days from each record's `completed_at` (see `webhook_activity` description in the response schema for the `pending`-status carve-out); sellers unable to honor that floor MUST omit the field entirely rather than truncate. When a buy has more historical fires than the limit, only the most recent are returned — there is no cursor for older fires; this surface is a debug aid, not a full audit log.", + "minimum": 1, + "maximum": 200, + "default": 50 + }, "pagination": { "$ref": "/schemas/core/pagination-request.json", "description": "Cursor-based pagination controls. Strongly recommended when querying broad scopes (for example, all active media buys in an account)." diff --git a/static/schemas/source/media-buy/get-media-buys-response.json b/static/schemas/source/media-buy/get-media-buys-response.json index caa9c7cac2..a660da7568 100644 --- a/static/schemas/source/media-buy/get-media-buys-response.json +++ b/static/schemas/source/media-buy/get-media-buys-response.json @@ -133,6 +133,14 @@ }, "uniqueItems": true }, + "webhook_activity": { + "type": "array", + "description": "Recent reporting and health webhook fires for the calling principal, most-recent first. Present only when `include_webhook_activity` was true in the request AND the seller surfaces this debug capability for this buy. Three-state semantics: (a) field omitted — seller does not surface webhook activity (either does not persist fire history, or `capabilities.media_buy.propagation_surfaces` excludes webhook surfaces, or the buy has no registered `push_notification_config` for this principal); (b) empty array `[]` — seller persists fire history but has fired nothing recent for this principal; (c) non-empty array — actual fire records. Sellers whose declared `propagation_surfaces` does not include `webhook` MUST omit the field. **Retention (normative):** sellers that surface this field MUST retain records for at least 30 days from each record's `completed_at` (for records still in `pending` status the clock runs from `fired_at` until the attempt terminates, then resets to 30 days from `completed_at` — so retry trails do not age out mid-flight). Sellers that cannot honor the 30-day floor MUST omit the field entirely rather than return a shorter window. Sellers MAY return fewer than `webhook_activity_limit` records when fewer fire records exist within the retention window. Sellers MUST emit one record per attempt — single-attempt successes appear as a single record with `attempt: 1`. Record shape is canonical across resources: see [`/schemas/core/webhook-activity-record.json`](/schemas/v3/core/webhook-activity-record.json) and snapshot-and-log.mdx § Webhook activity log pattern.", + "items": { + "$ref": "/schemas/core/webhook-activity-record.json" + }, + "maxItems": 200 + }, "history": { "type": "array", "description": "Revision history entries, most recent first. Only present when include_history > 0 in the request. Each entry represents a state change or update to the media buy. Entries are append-only: sellers MUST NOT modify or delete previously emitted history entries. Callers MAY cache entries by revision number. Returns min(N, available entries) when include_history exceeds the total.",