[Proposal] Introducing Webhook Event Handling for Platform Gateway

[ShakyaPr]

Problem Statement

API Platform Gateway currently does not have a dedicated webhook event handling path for events emitted by other platform components. This becomes a limitation when another component, such as Developer Portal or Management Portal, performs an operation asynchronously and Gateway needs to become aware of that operation later.

Examples:

• An API key is generated, regenerated, or revoked in Developer Portal.
• A subscription is created, changed, or deleted in Developer Portal.

Related Discussions : #1774

Scope

This effort focuses on the Gateway-side webhook implementation.

The first set of supported event families will be:

• API key events
  • apikey.generated
  • apikey.regenerated
  • apikey.revoked
• Subscription events
  • subscription.created
  • subscription.plan_changed
  • subscription.deleted

Proposal

• Introduce a webhook endpoint in Gateway Controller. External platform components will send signed HTTP POST requests to this endpoint. Gateway Controller will verify the request, decode the common event envelope, check idempotency, and dispatch the event to the matching event handler.
• The webhook endpoint is not intended to behave like a normal management API. A normal API request asks Gateway to perform an operation. A webhook event informs Gateway that an operation has already happened somewhere else, and Gateway should update its local state.

High-Level Flow

sequenceDiagram
participant DP as Developer Portal / Management Portal 
participant WH as Gateway Controller Webhook Endpoint 
participant REG as Event Handler Registry 
participant H as Event Handler 
participant DB as Gateway Storage 
participant XDS as xDS Snapshot Managers 
participant EH as EventHub 

DP->>WH: POST /devportal/events with signed event 
WH->>WH: Read raw body 
WH->>WH: Verify HMAC signature 
WH->>WH: Decode common event envelope 
WH->>WH: Check gateway_type 
WH->>WH: Check event_id idempotency 
WH->>REG: Dispatch by event_type 
REG->>H: Invoke matching handler 
H->>DB: Persist domain state 
H->>EH: Publish internal replica event 
EH->>XDS: Refresh policy engine state 
WH->>DP: Return 2xx when accepted

Loading

Webhook Endpoint

• Endpoint: POST /webhook/events
• Endpoint shoukd be configurable

[devportal_webhook] 
enabled = true 
secret = "5f74be8fdd5450a9793af3962f56da27f7de3f64263d73f739e9223f27cb1fd4" 
private_key_path = "/app/certificates/devportal-webhook.pem" 
gateway_type = "wso2/api-platform" 
signature_tolerance = "5m" 
max_body_size = "1MiB" 

Configuration options

• enabled - control whether the gateway webhook endpoint is registered
• secret - the HMAC signing secret shared with the event producer
• private_key_path - points to the GW private key used to decrypt encrypted sensitive payload data fields
• gateway_type - used to filter events meant for this gw type
• signature_tolerance - prevents replay atatcks using old signed requests
• max_body_size - protects endpoint from unexpectedly large request bodies

Event Envelope

• All the webhook events should follow common envelope format

{ 
"event_id": "550e8400-e29b-41d4-a716-446655440000", 
"event_type": "apikey.generated", 
"occurred_at": "2026-04-24T10:00:00.000Z", 
"org_id": "org-uuid", 
"gateway_type": "wso2/api-platform", 
"schema_version": "1.0", 
"aggregate_id": "domain-object-id", 
"data": {} 
}

• aggregate_id - Stable domain objecr ID affected by the event
• aggregate_id should be included for events where the affected domain object must be identified even when event-specific fields are missing or changed later.

Security Model

• Webhook security has two seperate responsibilities:
  1. Request authenticity & integrity
  2. Sensitive data confidentiality

Signature Verification

• Each webhook request must include an HMAC-SHA256 signature header.
  Ex: X-Devportal-Signature: t=1713952800,v1=<calculated_hmac>

• The signature should be calculated over the raw request body.

signed_payload = "<timestamp>.<raw_body>" 
signature = HMAC-SHA256(secret, signed_payload)

• GW controller should verify the req as follows:

1. Read the raw request body with a configured maximum size.
2. Extract t and v1 from the signature header.
3. Reject the request if the timestamp is outside the configured tolerance.
4. Compute the expected HMAC using the shared secret.
5. Compare the expected signature and received signature using a timing-safe comparison.
6. Reject the request if verification fails.

Sensitive Data Decryption

• Some events contain sensitive values, such as API keys or subscription tokens. These values should not be sent in plaintext.
• For those events, the producer should send an encrypted field such as:

{ 
"encrypted_key": 
   { 
       "wrappedKey": "<base64>", 
       "iv": "<base64>", 
       "tag": "<base64>", 
       "ciphertext": "<base64>" 
   } 
}

• Gateway Controller should decrypt this value using its configured private key.
• Note : The decrypted secret must only be kept in memory long enough to derive the required Gateway-side representation, such as a hash. It must not be logged or persisted in plaintext.

HTTP Processing Flow

flowchart TD
A[POST /devportal/events] --> B{Webhook enabled?} 
B -- No --> C[404] 
B -- Yes --> D[Read raw body with size limit] 
D --> E{Signature valid?} 
E -- No --> F[401 Unauthorized] 
E -- Yes --> G[Decode event envelope] 
G --> H{Envelope valid?} 
H -- No --> I[400 Bad Request] 
H -- Yes --> J{gateway_type matches?}
J -- No --> K[202 Accepted as no-op] 
J -- Yes --> L{Dispatch by event_type} 
L --> M[Apply event-specific DB idempotency check]
M --> N{Domain state already persisted?}
N -- Yes --> O[Skip duplicate DB write]
N -- No --> P[Persist domain state]
O --> Q[Publish EventHub event for replicas]
P --> Q
Q --> R{Publish successful?}
R -- No --> S[500 Retryable failure]
R -- Yes --> T[Refresh policy engine state using xDS]
T --> U{Refresh successful?}
U -- No --> V[500 Retryable failure]
U -- Yes --> W[200 OK or 202 Accepted]

Loading

Webhok vs Normal GW API

Area	Normal GW API	Webhook Endpoint
purpose	Command Gateway to perform an operation	Notify Gateway about an operation that already happened
Caller	client, CLI, UI or CP	Trusted platform components such as Devportal
Auth	Basic auth, JWT or IDP based	HMAC signature with shared secret
Processing model	usually syncronous	Asynchronous event handling
Idempotency	Useful but not always mandatory	Required as it is possible to happen retry events
Consistency model	Req-res consistency	Eventual consistency
Extension model	Add new API Op	Add new event handler

Suggested Package Structure for Development

gateway/gateway-controller/pkg/webhook/
  config.go
  envelope.go
  server.go
  registry.go
  verifier.go
  decryptor.go
  idempotency.go
  errors.go

gateway/gateway-controller/pkg/webhook/handlers/
  apikey.go
  subscription.go
  ...

API Key Event Handling

Supported Events

Event	Meaning
apikey.generated	A new API Key was generated for an API outside the Gateway
apikey.regenerated	AN exisitng API key was rotated outside GW
apikey.revoked	An API key was revoked outdside GW

API Key Generation Event Payload

{ 
    "event_id": "event-uuid", 
    "event_type": "apikey.generated", 
    "occurred_at": "2026-04-24T10:00:00.000Z", 
    "org_id": "org-uuid", 
    "gateway_type": "wso2/api-platform", 
    "schema_version": "1.0", 
    "aggregate_id": "key-uuid",     # Optional
    "data": { 
           "key_id": "key-uuid",        // Uses for external_ref_id, required field
           "name": "my-key", 
           "expires_at": "2027-01-01T00:00:00.000Z", 
           "api": { 
                    "ref_id": "api-handle",          // required for api key generation to identify the API related to generated key
                    "name": "Order API", 
                    "version": "1.0.0" 
             }, 
            "subscription": {                    // Though subscription object is sent in the event, still there's no usage when processing event
                     "ref_id": "subscription-uuid", 
                     "plan_name": "Gold" 
            }, 
            "encrypted_key": { 
                     "wrappedKey": "<base64>", 
                     "iv": "<base64>", 
                     "tag": "<base64>", 
                     "ciphertext": "<base64>" 
            } 
    } 
}

API Key Revoke Event Payload

{
    "event_id": "event-uuid", 
    "event_type": "apikey.revoked", 
    "occurred_at": "2026-04-24T10:00:00.000Z", 
    "org_id": "org-uuid", 
    "gateway_type": "wso2/api-platform", 
    "schema_version": "1.0", 
    "aggregate_id": "key-uuid",     # Optional
    "data": {
        "key_id": "key-uuid",          // required
        "name": "my-key",
         "api": {
              "name": "MyFlights API",
              "version": "1.0.0",
               "ref_id": "api_handler"
         }
  }

API Key Regenerate Event Payload

{
    "event_id": "event-uuid", 
    "event_type": "apikey.regenrated", 
    "occurred_at": "2026-04-24T10:00:00.000Z", 
    "org_id": "org-uuid", 
    "gateway_type": "wso2/api-platform", 
    "schema_version": "1.0", 
    "aggregate_id": "key-uuid",     # Optional
    "data": {
        "key_id": "key-uuid",             // required
        "name": "my-key",
         "api": {
              "name": "MyFlights API",
              "version": "1.0.0",
               "ref_id": "api_handler"
         }
  }

• data.key_id : This will be mapped to external_ref_id of api_keys table, useful to
  • identify whether the api key has been already generated when api_key.generated event receives
  • select the correct api key when api_key.regenerate or api_key.revoked event receives

API Reference Resolution

• data.api.ref_id should resolve to Gateway artifact handle.
• The webhook handler should use the handle to find the API artifact in GW persistence storage

API Key Generation Flow

1. Resolve the API using data.api.ref_id
2. Decrypt data.encrypted_key
3. Hash the decrypted key using Gateway API key hashing rules.
4. Clear the plaintext secret from memory as soon as possible
5. persist or upsert the API ket in GW storage
6. Update the API key state used by the policy engine.
7. Publish an internal EventHub event so other Gateway Controller replicas update their local state.

API Key Regeneration/Revoke

• The flow is similar to generation, but uses data.key_id which maps to external_ref_id of GW database to identify the exact API key.

Subscription Event Handling

Supported Events

Event	Meaning
susbcription.created	A subscription was created outside Gateway.
subscription.plan_changed	A subscription plan was changed outside Gateway.
subscription.deleted	A subscription was deleted outside Gateway.

Subscription Creation Event Payload

{ 
    "event_id": "event-uuid", 
    "event_type": "subscription.created", 
    "occurred_at": "2026-04-24T10:00:00.000Z", 
    "org_id": "org-uuid", 
    "gateway_type": "wso2/api-platform", 
    "schema_version": "1.0", 
    "aggregate_id": "subscription-uuid",              // Optional
    "data": { 
           "subscription_plan": { 
                    "ref_id": "subscription_plan_name",          // required, since org+plan_name unique in DB
                    "status": "ACTIVE"
             }, 
            "api": { 
                     "ref_id": "api_handle",                    // required
                     "name": "Order API" 
                     "version": "1.0.0"
            }, 
            "encrypted_key": { 
                     "wrappedKey": "<base64>", 
                     "iv": "<base64>", 
                     "tag": "<base64>", 
                     "ciphertext": "<base64>" 
            } 
    } 
}

Subscription Deletion Event Payload

{
  "event_id": "event-uuid", 
  "event_type": "subscription.deleted", 
  "occurred_at": "2026-04-24T10:00:00.000Z", 
  "org_id": "org-uuid", 
  "gateway_type": "wso2/api-platform", 
  "schema_version": "1.0", 
  "aggregate_id": "subscription-uuid",              // Optional
  "data": {
    "subscription_plan": {
      "ref_id": "subscription_plan_name",
      "plan_name": "Gold",                  // Optional
      "status": "ACTIVE"                     // Optional
    },
    "api": {
      "name": "MyFlights API",
      "version": "1.0.0",
      "ref_id": "api_handle"
    }
  }
}

Subscription Reference Resolution

• data.api.ref_id should resolve GW API handle
• data.subscription_plan.ref_id should resolve subscription plan name for the provided org
• Uses both ref ids to identify the subscription in the persistence storage

Subscription Creation Flow

1. Resolve the API using data.api.ref_id
2. Resolve the sub plan using data.susbcription_plan.ref_id
3. If only these two available, decrypt the susbcription token from data.encryted_key
4. Hash the token
5. Resolve the uuids of api & suscription plan using the ref_ids from DB
6. Checks whether an entry with uuids exists & skip persistence if already exists
7. Persist or upsert the sub as ACTIVE
8. Publish an Eventhub event
9. Refresh the susbcription xDS snapshot

Subscription Plan Update/Delete

• Same flow, but uses ref_ids to identify the subscription.

Persistence and State Update Order

flowchart LR
A[Validate event] --> B[Apply domain change to storage] 
B --> C[Publish EventHub event] 
C --> D[Refresh xDS snapshot] 
D --> E[Mark webhook event as processed]

Loading

Idempotency

• Webhook delivery is retry-based. The same event may be delivered more than once if Developer Portal does not receive a successful response or if the request times out. Gateway must therefore handle duplicate deliveries safely.
• The handler for each event type should check whether the requested domain change has already been applied in Gateway storage.
• If the state already exists, the handler should skip the duplicate persistence step and continue with the remaining steps, such as policy engine refresh and EventHub publishing.
• This is important because the previous attempt may have succeeded in persisting the data but failed before completing xDS refresh or EventHub publishing. In such cases, returning 500 causes Developer Portal to retry the same event.
• On retry, Gateway should detect that the domain state is already present and continue from the remaining steps.
• If the Gateway operation sucess, but returning status code fails due to connectivity issue, then retry will happen but DB operation will be skipped while performing xDS snapshot refresh & eventHub publishing.

sequenceDiagram
participant DP as Developer Portal 
participant GW as Gateway Controller 
participant DB as Gateway Storage 
participant XDS as xDS Snapshot Manager 
participant EH as EventHub 

DP->>GW: POST event 
GW->>GW: Verify signature and validate envelope 
GW->>DB: Check event-specific domain state 
alt State not found 
      GW->>DB: Persist domain state 
else State already exists 
       GW->>GW: Skip duplicate DB write 
end 
GW->>XDS: Refresh policy engine state 
alt xDS refresh fails 
       GW-->>DP: 500 Retryable failure 
else xDS refresh succeeds 
       GW->>EH: Publish internal replica event 
       alt EventHub publish fails 
                   GW-->>DP: 500 Retryable failure 
       else EventHub publish succeeds 
                   GW-->>DP: 200 OK 
       end 
end

Loading

EventHub & Replica Synchronization

• After a webhook handler successfully updates local storage, it should publish an internal EventHub event. This allows other Gateway Controller replicas to refresh their local in-memory state and xDS snapshots.

flowchart LR
A[External webhook event] --> B[Receiving Gateway Controller] 
B --> C[Update shared storage] 
C --> D[Publish internal EventHub event] 
D --> E[Other Gateway Controller replicas] 
E --> F[Refresh local state and xDS snapshots]

Loading

Error Handling

Case	Response	Notes
Webhook disabled	404
Missing signature	401	Request is unauthenticated
Invalid Signature	401	Do not reveal detauils
Timestamp outside tolerance	401	Possible replay
Invalid JSON	400	Bad Request
Missing required envelope field	400	Bad Request
Unsupported event type	400	Bad Request
Gateway type mismatch	202	Accepted as no-op
Duplicate processed event	200	Idempotent success
Temporary storage failure	500	Producer should retry
Temporary xDS update failure	500	Producer should retry

[ShakyaPr]

Open Questions

1. Should we handle multi-replica deployment related idempotency using the suggested approach or should we persist the processed events in gateway storage to share between replicas?
2. How should we handle idempotency cache TTL? Through a Background cleanup goroutine or mechanism to check expired idempotency caches when updating cache for a new processed event?
3. How partial failures should handle? If the gateway stirage persistence success but if the xDS publish or EventHub publish fails, should we need to retry? or return 500 and ask for retry? In this case how should the gateway storage persistence handles? Do we rollback it or continue the next retry from the latest state?
4. Should subscription deletion physically delete the row or mark it inactive?

[ShakyaPr]

Open QUestion Answers

• For Q1 & Q2, idempotency will be handled throughDB persistence, use DB to identify the event has been already persisted or need to perform db operation depending on the event type. No need to have an idempotency cache & synchronization between replicas. IF the event has been processed but retun response fails, retry will be happened and the other than DB operation other two steps will be performed again.
• For Q3, it will be marked as 500 internal server error & retry will be happened.
• For Q4, subscription will be deleted

[ShakyaPr]

This feature will not be implemented & instead, the webhook event handling will be supported via Platform API. Therefore, going forward, the same connection between GW & CP (Websocket) will be used to propagate Devportal changes to GW by emitting events to Platform API webhook endpoint.

Hence, close the discussion.