Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Implement push notifications (webhooks) #70

Closed
mjallday opened this Issue Sep 13, 2012 · 44 comments

Comments

Projects
None yet
Contributor

mjallday commented Sep 13, 2012

It would be great if Balanced could push notifications when certain criteria are met. Foremost in my mind would be a notification when the state of a credit is updated.

Contributor

mahmoudimus commented Sep 13, 2012

+1. Would webhooks work? Are they durable? What's the retry time? Should we use response codes to signal retry logic?

Contributor

mjallday commented Sep 13, 2012

Webhooks seem fine with a signed payload. They should keep retrying until a 2XX response code is returned.

Retry logic should be try, on failure wait 1min, on failure wait 2min, on failure wait 4min...

+1

Please implement webhooks first and foremost on the most intensive requests:

  1. create an account
  2. create a hold
  3. capture a hold

In my testing, the combination of 1+2 currently takes ~3 seconds in our production system. This is pretty unacceptable in any kind of scale.

We are planning to implement our own solution based on workers (especially for 1+2). If you guys have something coming soon, we will probably wait.

Contributor

ajsharp commented Oct 24, 2012

+1 to webhooks.

+1

It's going to cause a lot of headaches to manually track down payment failures and sort them out. Instead of that happening programatically and notifying the user to update their bank info, we're going to have to have people who receive the payment failure emails and then manually track down the user, try to get the accurate bank info and then manually issue payouts. This could all be solved with webhooks.

Owner

matin commented Nov 30, 2012

A marketplace pointed out the other day that they're unable to initiate transaction in the dashboard because it would put them out of sync with their own database. It would be impossible to poll every transaction. This means the marketplace can only use the dashboard in read mode until webhooks are available.

bcm commented Nov 30, 2012

assuming Copious isn't the marketplace you're describing, then I'll mention
that's essentially our situation as well.

although, I'm not sure to what extent a web hook telling us about a new
transaction would be useful, since we wouldn't know which order in our
system that transaction would be related to without some standardized way
for the web hook to communicate that to us (and by implication, for the
person initiating the transaction in the dashboard to be able to specify
that information).

On Fri, Nov 30, 2012 at 8:59 AM, Matin Tamizi notifications@github.comwrote:

A marketplace pointed out the other day that they're unable to initiate
transaction in the dashboard because it would put them out of sync with
their own database. It would be impossible to poll every transaction. This
means the marketplace can only use the dashboard in read mode until
webhooks are available.


Reply to this email directly or view it on GitHubhttps://github.com/balanced/balanced-api/issues/70#issuecomment-10896021.

Owner

matin commented Nov 30, 2012

@bcm I was referring to Copious. I sat next to Karen and watched her use the dashboard two days ago.

although, I'm not sure to what extent a web hook telling us about a new
transaction would be useful, since we wouldn't know which order in our
system that transaction would be related to without some standardized way
for the web hook to communicate that to us (and by implication, for the
person initiating the transaction in the dashboard to be able to specify
that information).

Noted.

This would be a huge win. I'd personally copy what stripe did(https://stripe.com/docs/webhooks). +1

joonas commented Dec 3, 2012

👍 from me

+1

+1

Contributor

ajsharp commented Dec 28, 2012

I'm going to use this comment to provide feedback on the problems that we have at Zaarly, and how we'd like to use webhooks to solve those problems.

For our purposes, we'd like to see webhooks implemented primarily to stay current with lifecycle events for transactions (credits, debits, holds, refunds), and secondarily for other types of Balanced resources (accounts).

Our use cases can be summarized as:

  1. Updates to existing objects (credit state updates, debit states for the marketplace bank account debits)
  2. Creation of new objects (refunds et al created via the balanced dashboard)

From the client system perspective, webhooks are a solution to keep the client system in sync with Balanced. This is a critical difference from many webhook APIs, which often serve secondary purposes, such as analytics events.

Synchronization is an essential yet difficult-to-implement detail. One might summarize this issue as a gateway problem: Assuming that the client system is doing some sort of internal accounting, it's critical that your system have the ability to stay in sync when new objects are created and updates are made to existing objects outside of your system (that don't go through your gateway). Updates and inserts to these objects need to eventually go through your gateway in order to keep your system's records in good order.

Webhooks are a great way to do this, particularly because they happen in semi-real-time, and the nature of a request-response protocol like http is well-suited to communicating success or failure of the messages passed from server-to-client.

A batch approach on the client-side is probably still necessary to do sanity checks, but becomes much less critical in the synchronization portion of the system. This is a good thing, because batch systems tend to be pretty error-prone, and getting the time windows correct for updates to existing objects has some fairly apparently holes in it.

The webhook system that Poundpay had was fantastic, so I hope we can get something similar in Balanced soon :)

Fantastic write-up Alex.

Contributor

ajsharp commented Dec 28, 2012

thx @taylorbrooks 😄

@matin matin referenced this issue Dec 28, 2012

Closed

Implement events #237

Oh snap! UI for webhooks?! Drool.

Contributor

ajsharp commented Jan 11, 2013

OH SON.

Contributor

mahmoudimus commented Jan 16, 2013

A beta version of webhooks has been released to the dashboard. Let us know what you think. We're collecting feedback on the interaction.

Right now, it's for ONE type of event credit.updated which I'll highlight more on issue #237

I will discuss events, their purpose, and what data is needed to represent an event more on that issue as well as enumerate a list of possible events.

/cc @taylorbrooks @zealoushacker @ajsharp @Breefield @evangoldin @bcm @stevewanless @jeregrine @joonas @twanlass @phallguy

Contributor

mahmoudimus commented Jan 16, 2013

The main issue with webhooks is testing and implementing them. Balanced will not consider this issue closed until we've done an awesome job showing you how to test and implement webhooks with best practices.

So, this is something to keep in mind as we proceed -- a large majority of developers are not familiar with asynchronous programming.

The big events are charge failed, charge succeeded, charge disputed. I like
to use those as kicking off emails for notifications, receipts and
disabling accounts etc.

Contributor

mahmoudimus commented Feb 13, 2013

webhooks!

Updates, check this out: https://www.balancedpayments.com/docs/api-preview?language=bash#events

We're looking for feedback before we polish it.

Nice! Only thing that might be helpful is to show example data sent to the
webhook.

On Wed, Feb 13, 2013 at 1:59 PM, Mahmoud Abdelkader <
notifications@github.com> wrote:

webhooks!

Updates, check this out:
https://www.balancedpayments.com/docs/api-preview?language=bash#events

We're looking for feedback before we polish it.


Reply to this email directly or view it on GitHubhttps://github.com/balanced/balanced-api/issues/70#issuecomment-13514785.

Contributor

mjallday commented Feb 15, 2013

We just merged #254 so callbacks and events are now officially exposed. You can view the callbacks and events resources to see what the payloads look like. We've also made documentation.

You can add these to your marketplace via the settings tab on the Balanced dashboard if you don't want to use the client libraries to do so and events are viewable via the dashboard for transactions. You can also replay an event to a selected callback if you're debugging/developing.

The python and ruby client libraries have support for adding, and removing callbacks and browsing events (PHP and Java libraries will be updated soon). You can see an example from the python library of how to use callbacks (or ruby) with the client libraries.

Let us know if you'd like to see examples of how to consume these events, we use them internally for all sorts of stuff so we've developed some patterns that may be handy. 🎱

A bug? Something missing? Let us know by commenting in this thread!

@mjallday mjallday closed this Feb 15, 2013

Are the webhooks signed? How can we be certain they're authentic? Thank you!

Contributor

mjallday commented Feb 19, 2013

Great question, the simplest approach is to grab the URI of the resource that comes back, do a GET/find operation on the resource which will retrieve it directly from the Balanced servers.

That workflow should be documented somewhere if the webhooks themselves won't be signed. And I still think signing is a good extra level of assurance. A SHA-256 of the user's API key and the webhook body would work but it would be even better to sign using a timestamp as well to avoid replay attacks. Either way, the onus is on the receiver to verify.

@mahmoudimus mahmoudimus referenced this issue Feb 22, 2013

Closed

ACH Debit #2

Contributor

ajsharp commented Feb 23, 2013

That's a great point. Signature verification would be a great thing to have
built into the balanced client libraries.

On Tue, Feb 19, 2013 at 11:45 AM, Steve Richert notifications@github.comwrote:

That workflow should be documented somewhere if the webhooks themselves
won't be signed. And I still think signing is a good extra level of
assurance. A SHA-256 of the user's API key and the webhook body would work
but it would be even better to sign using a timestamp as well to avoid
replay attacks. Either way, the onus is on the receiver to verify.


Reply to this email directly or view it on GitHubhttps://github.com/balanced/balanced-api/issues/70#issuecomment-13790331.

Cheers,

alex sharp

https://twitter.com/ajsharp
http://alexjsharp.com

I would love to be able to look at a signature (SHA of API key in the header) before inspecting the data. Especially since these are financial transactions that we are inspecting and responding to.

Contributor

mjallday commented Jun 19, 2013

@camstorrs that's a feature we'll get to. In the meantime you can grab the URI of the resource that comes back, do a GET/find operation on the resource which will retrieve it directly from the Balanced servers to ensure that authenticity of the payload.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment