Webhook deliveries show "completed" even when they fail #1949
Replies: 3 comments
-
|
Faced a similar issue. None of the webhooks are working right now. The problem started with this PR: #1903 With the timeouts, nothing gets resolved. Not even localhost! |
Beta Was this translation helpful? Give feedback.
-
|
Hey all, The cause of the delivery failures is that Resolv::DNS started returning no results for every query, the exact root cause of that is as of yet unknown. As for the UI improvements, I'll work on those next week. We should surface the contents of the result object so that you can know what status code the webhook encountered and I'll think about combining the error and failed states. The idea behind the state is to tell you if your webhook was processed, while the result should tell you what happened if it succeeded. |
Beta Was this translation helpful? Give feedback.
-
|
I’ve run into similar webhook issues in production before. Everything showed as “completed”, but the events never reached the destination. Using Sigryn as an intermediate layer helped a lot, since it gives real visibility into webhook delivery, attempts, and failures instead of relying only on the provider’s status. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
Webhook deliveries on fizzy.do show "completed" status but requests never arrive at the destination. Tested with three different endpoints (Basecamp Chat, RequestBin, webhook.site). None received any requests despite all showing "completed" in the Fizzy UI.
Steps to reproduce
What I expected
The webhook request to arrive at the destination, or if it failed, a clear indication of why.
What actually happened
I verified curl to the same Basecamp chatbot URL works fine from my machine, so the destination is accepting requests.
Investigation
I looked through the code to understand what might be happening:
The
statefield is misleadingIn
Webhook::Delivery, the state machine has:pending,in_progress,completed,errored.The
completedstate means "the delivery job finished" - not "the request succeeded." Looking atdeliver:The actual outcome is stored in
response, which can be:{ code: 200 }: success{ error: :private_uri }: SSRF protection blocked it{ error: :dns_lookup_failed }: DNS resolution failed{ error: :connection_timeout }: request timed outThe model has
succeeded?andfailed?methods that check this, but the UI only displaysstate.No HTTP request may be made at all
In
perform_request:If
SsrfProtection.resolve_public_ipreturns nil, no HTTP request is ever made. This happens if DNS resolution fails, times out (2 second limit), or returns IPs that get filtered.Since 3 different public destinations all failed to receive requests, this suggests a systematic issue rather than a destination-specific one.
Questions
What's in
delivery.response? This would reveal why requests are failing. I can't check this on fizzy.do since I don't have database access.Is there a known issue with outbound webhooks on fizzy.do? The pattern (all destinations fail, "completed" status shows) suggests infrastructure-level DNS or network config issue.
Suggested improvements
Regardless of root cause, the UI should surface delivery outcomes more clearly:
delivery.succeeded?rather than justdelivery.stateresponse[:code](e.g., "HTTP 200") orresponse[:error](e.g., "Connection timeout")The model already has
succeeded?andfailed?methods, but the UI just doesn't use them.I'm happy to submit a PR for the UI fix once the underlying delivery issue is understood.
Beta Was this translation helpful? Give feedback.
All reactions