Skip to content
This repository was archived by the owner on Sep 20, 2023. It is now read-only.

Queue for dshop backend processes#4465

Merged
nick merged 12 commits intomasterfrom
danielvf/dshop-queue
May 19, 2020
Merged

Queue for dshop backend processes#4465
nick merged 12 commits intomasterfrom
danielvf/dshop-queue

Conversation

@DanielVF
Copy link
Copy Markdown
Contributor

@DanielVF DanielVF commented May 16, 2020

When a user makes a Stripe payment, we now enqueue the work of creating an offer and submitting it to the blockchain.

This PR uses the bull queue package, which is backed by redis. It seems to have a good API for this, doesn’t try to do too much, and has a lot of built in error handling.

It is great to be able to run our stack without external dependencies, so I’ve written a fallback queue class that is used if redis is not configured. In this case, we skip the queue, and instead process queue items as soon as they are added.

I’ve added bull-board UI to /super-admin/queue

In the future, we'll want to also queue up the actions that run on the listener side. I wanted to get this PR live first to verify our approach, and get our education before adding those. Those should be very quick.

DanielVF added 3 commits May 16, 2020 09:47
- Queue submitting offers to the blockchain from stripe payments
— Includes a UI for viewing queue items
- Uses redis via bull, but can fall back to not using any queue.

Part of issue #16
@DanielVF DanielVF requested review from mikeshultz and nick May 18, 2020 11:27
Comment thread dapps/shop/backend/package.json
Comment thread dapps/shop/backend/app.js Outdated
Comment thread dapps/shop/backend/queues/makeOfferProcessor.js Outdated
Comment thread dapps/shop/backend/queues/makeOfferProcessor.js Outdated
@DanielVF DanielVF changed the title [Draft] Queue for dshop backend processes. Queue for dshop backend processes May 18, 2020
Comment thread dapps/shop/backend/queues/queues.js
@DanielVF DanielVF added the dshop label May 18, 2020
Comment thread dapps/shop/backend/queues/makeOfferProcessor.js
Copy link
Copy Markdown
Contributor

@mikeshultz mikeshultz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Minor commentary inline.

Unfortunately, I’ve not been able to get an end-to-end shop running locally with Stripe. I’ve tested this PR by using the stripe webhook tool, and overriding the data that is passed from stripe in the webhook route. I’ve seen the queue items being created and processing started, but it’s not end to tested.

For future reference, they have a handy docker container you can use for sandbox testing:

docker run --rm -it stripe/stripe-cli --api-key "sk_test_XXXXxxxXXXX" listen -jf https://localhost:3000/webhook --skip-verify

.catch(err => {
console.log(err)
})
{ attempts: 6 }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 6?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to give it a longer time window to try over. Other than that, it's arbitrary.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a dealbreakr or antyhing, but you could configure backoff too.

* job.data should have {shopId, amount, encryptedData}
* @param {*} job
*/
async function processor(job) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might recommend using job.progress to show completion. It can sometimes help with troubleshooting, though perhaps job.log takes care of that here.

Comment thread dapps/shop/backend/queues/queues.js
@nick
Copy link
Copy Markdown
Contributor

nick commented May 18, 2020

It turns out that the Bull v4 API is different from v3. I think the UI also only works with v3 when you use the alpha build.

Probably easiest to just downgrade to v3 for now?

@DanielVF
Copy link
Copy Markdown
Contributor Author

Moved back to bull 3, and this thing is actually tested now! Give it a whir.

@nick nick merged commit b0d6f20 into master May 19, 2020
@nick nick deleted the danielvf/dshop-queue branch May 19, 2020 16:48
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants