Skip to content

fix: stop buy-crypto transactions for blocked/deleted users to unblock batch completion#3680

Merged
TaprootFreak merged 6 commits into
developfrom
fix/unblock-batch-blocked-user
May 7, 2026
Merged

fix: stop buy-crypto transactions for blocked/deleted users to unblock batch completion#3680
TaprootFreak merged 6 commits into
developfrom
fix/unblock-batch-blocked-user

Conversation

@TaprootFreak
Copy link
Copy Markdown
Collaborator

@TaprootFreak TaprootFreak commented May 7, 2026

Problem

A race condition in the buy-crypto payout pipeline causes permanent deadlocks when a user is deleted or blocked after their transaction has been added to a payout batch. The same issue affects the fiat-output pipeline (sell-crypto).

Incident: Monero pipeline blocked for 18+ hours

On 2026-05-06, a single deleted user blocked the entire XMR payout pipeline:

Time Event
17:26:36 BuyCrypto 121819 created (67 EUR → 0.1769 XMR, user 399153, userData 392122)
17:41:00 User 399153 deleted (between transaction creation and payout)
17:44:15 Batch 61024 created, both TX 121817 and 121819 set to ReadyForPayout
17:44:17 Payout loop: TX 121817 (user active) → PayoutOrder created ✅
17:44:17 Payout loop: TX 121819user.isBlockedOrDeleted = truethrow Error → caught → skipped
17:48:20 TX 121817 completes normally
18+ hours TX 121819 stuck in ReadyForPayout, no PayoutOrder ever created

Result: 33 XMR transactions (~2,730 EUR) from different users were blocked and could not be processed.

Root cause (buy-crypto)

1. payoutTransactions() detects blocked user → throws error → caught → continues
   → No PayoutOrder created, transaction stays in ReadyForPayout

2. checkCompletion() calls checkOrderCompletion() for TX without PayoutOrder → throws
   → tx.isComplete stays false

3. Batch completion requires: batch.transactions.every(tx => tx.isComplete)
   → batch stays in PayingOut → per-asset lock blocks ALL new batches

4. All new XMR transactions stay in Created status indefinitely

Root cause (fiat-output)

The same pattern exists in setReadyDate(): a blocked user causes a throw that is caught and logged every minute, without ever resolving the fiat output. The linked BuyFiat entities never get their amlCheck set to Fail, preventing the chargeback process from returning the crypto.

Solution

1. stop() method on BuyCrypto entity with AML reason

Sets status = STOPPED and optionally amlCheck = Fail with the appropriate AmlReason:

  • USER_DELETED — user account deleted
  • USER_BLOCKED — user account blocked
  • USER_DATA_BLOCKED — userData blocked or risk-blocked

Does not set isComplete — reserved exclusively for confirmed on-chain payouts.

2. Buy-crypto payout: stop and trigger chargeback (prevention + recovery)

  • Payout loop: Stops blocked/deleted user transactions immediately with amlCheck = Fail
  • checkCompletion(): Detects and stops stuck transactions in existing batches
  • Batch completion: Accepts STOPPED transactions alongside isComplete

3. Fiat-output: set amlCheck = Fail on BuyFiat entities (sell-crypto chargeback)

Instead of throwing (and repeating every minute), the blocked user's linked BuyFiat entities get amlCheck = Fail with the appropriate reason. This triggers the existing sell-crypto chargeback flow to return the crypto.

4. Consistency: use entity method in transaction service

The existing TransactionService.stop() set status directly. Updated to use the entity stop() method.

Files changed

File Change
buy-crypto.entity.ts stop() accepts optional amlReason, sets amlCheck = Fail
buy-crypto-out.service.ts Blocked user handling with AML reason in payout loop + checkCompletion, batch completion accepts STOPPED
fiat-output-job.service.ts Set amlCheck = Fail on BuyFiat for blocked users instead of throwing
transaction.service.ts Use entity stop() method

Verified: no side effects

  • Notification service: Queries filter on status: COMPLETE or amlCheck: PENDING — no false matches
  • Preparation service: Already excludes STOPPED in refreshFee() and fillPaymentLinkPayments(); amlCheck = Fail prevents re-processing
  • History/DTO mapping: Already maps STOPPED to TransactionState.STOPPED

Test plan

  • Deploy to DEV
  • Verify batch 61024 completes automatically (TX 121819 gets stopped with amlCheck = Fail)
  • Verify the 33 pending XMR transactions start flowing through the pipeline
  • Verify stopped transactions have amlCheck = Fail with correct amlReason
  • Verify normal (non-blocked user) payout flow is unaffected
  • Verify fiat-output blocked users get amlCheck = Fail on linked BuyFiats
  • Verify isComplete remains false for stopped transactions

…k batch completion

When a user is deleted or blocked after their transaction enters a
payout batch, the batch gets permanently stuck in PayingOut status.
The blocked user check throws an error that is caught and silently
continued, but the transaction never receives a payout order and can
never complete. Since batch completion requires ALL transactions to
have isComplete=true, one stuck transaction blocks the entire batch
indefinitely. The per-asset batch lock then prevents any new batches
for the same output asset from being created.

Changes:
- Add stop() method to BuyCrypto entity (sets STOPPED + isComplete)
- In payoutTransactions(): stop blocked/deleted user transactions
  instead of throwing, preventing the dead-end from being created
- In checkCompletion(): detect and stop blocked/deleted user
  transactions to unblock existing stuck batches
…tion check

isComplete should only be true when the payout is confirmed on-chain.
The batch completion check now also accepts STOPPED transactions,
so stopped transactions no longer block batch progression.
…ency

The existing manual stop in transaction.service.ts set the status
directly instead of using the entity method pattern. Use the new
stop() method for a single source of truth.
@TaprootFreak TaprootFreak marked this pull request as ready for review May 7, 2026 09:12
@TaprootFreak TaprootFreak requested a review from davidleomay as a code owner May 7, 2026 09:12
…back

When a user is blocked or deleted, the money must be returned via the
existing chargeback process. Set amlCheck=Fail with the appropriate
AmlReason (USER_BLOCKED, USER_DATA_BLOCKED, or USER_DELETED).

Buy-crypto: stop() now accepts optional amlReason parameter.
Fiat-output: set amlCheck=Fail on linked BuyFiat entities for blocked
users instead of throwing, triggering the sell-crypto chargeback flow.
@TaprootFreak TaprootFreak merged commit 854b6f3 into develop May 7, 2026
7 checks passed
@TaprootFreak TaprootFreak deleted the fix/unblock-batch-blocked-user branch May 7, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants