Skip to content

fix(payout): recover EVM payouts from out-of-gas reverts#3697

Merged
TaprootFreak merged 6 commits into
developfrom
fix/evm-payout-oog-recovery
May 12, 2026
Merged

fix(payout): recover EVM payouts from out-of-gas reverts#3697
TaprootFreak merged 6 commits into
developfrom
fix/evm-payout-oog-recovery

Conversation

@TaprootFreak
Copy link
Copy Markdown
Collaborator

@TaprootFreak TaprootFreak commented May 12, 2026

Summary

Closes the failure mode where an EVM payout TX runs out of gas on-chain and the order stays in PayoutPending indefinitely.

Root cause

Two defects had to coincide:

  1. Failure swallowed. isTxComplete threw on reverted receipts. The outer try/catch in PayoutService.checkPayoutCompletion logged the error every 30 s but left the order in PayoutPending.
  2. No retry path for mined reverts. canRetryFailedPayout only handles mempool-expired TXs (isTxExpired); a mined receipt with status=0 is not expired and thus not retryable.

The underlying race itself — gas estimate vs execution diverging because of an EIP-2200 SSTORE swing on the recipient's balance slot — is rare in practice; the failure mode that needs fixing is the stuck order, not the rare gas estimate miss.

Fix

  • getPayoutCompletionData — returns a tri-state PayoutTxStatus (pending / complete / failed with isOutOfGas) read directly from the receipt instead of relying on the throwing isTxComplete.
  • EvmStrategy.checkPayoutCompletionData — handles failed explicitly:
    • isOutOfGasrollbackPayout() → next cron pass retries with a fresh nonce and a re-estimated gas limit.
    • non-OOG revert → designatePayout()processFailedOrders already mails and moves the order to PayoutUncertain.
  • PayoutOrder.rollbackPayout() — new entity method symmetric to pendingPayout(txId): clears payoutTxId and resets status to PreparationConfirmed. Unit test added.
  • sendToken now passes the real transfer amount through to getTokenGasLimitForContact (consistent with prepareTxData).
  • Redundant getPayoutCompletionData overrides in base-{coin,token}, gnosis-{coin,token}, optimism-{coin,token}, polygon-{coin,token} strategies removed — all 18 EVM strategies now consistently use the parent implementation.
  • PayoutTxStatus collocated with PayoutRequest / FeeResult in payout/interfaces/index.ts.

No DB migration, no new status, no new entity columns, no change to gas estimation values — everything routes through existing entity methods (rollbackPayout, designatePayout, pendingInvestigation) and the existing processFailedOrders mail+investigation path.

Test plan

  • CI green (format / type-check / lint / jest)
  • Manual: trigger an EVM payout that previously failed, verify auto-rollback + retry produces a successful TX with a new nonce
  • Manual: simulate a non-OOG revert (e.g. paused token), verify order ends in PayoutUncertain with mail

Three issues combined could leave an EVM payout order stuck in
PayoutPending indefinitely:

1. Gas estimation buffer (x1.2) was too small to absorb the
   EIP-2200 SSTORE swing (~15k gas) that occurs when the recipient's
   token balance slot changes between estimate-time and execution-time
   (cold zero-to-nonzero costs 22.1k vs warm nonzero-to-nonzero 7.1k).
2. isTxComplete threw on reverted receipts, leaving the order
   silently in PayoutPending while the cron logged errors every 30s.
3. canRetryFailedPayout only handled mempool-expired transactions,
   not mined-but-reverted ones.

Changes:
- getTokenGasLimitForContact uses a 50% buffer plus an 80k floor;
  estimation failures now fall back to 150k. sendToken passes the
  real transfer amount through so the estimate reflects execution.
- getPayoutCompletionData returns a tri-state PayoutTxStatus
  (pending / complete / failed with isOutOfGas) instead of
  throwing on revert.
- EvmStrategy.checkPayoutCompletionData treats reverted out-of-gas
  as recoverable (rollbackPayoutDesignation + clear payoutTxId so
  the next cron pass retries with a fresh nonce and re-estimated
  gas) and non-OOG reverts as needing investigation
  (designatePayout, picked up by processFailedOrders).
- Redundant getPayoutCompletionData overrides removed from
  base/gnosis/optimism/polygon coin+token strategies.
Encapsulate the payout-state rollback in a dedicated entity method
symmetric to pendingPayout(txId), instead of misusing
rollbackPayoutDesignation() (intended for PayoutDesignated state)
and mutating payoutTxId directly from the strategy.
Keep cross-file types collocated with PayoutRequest and FeeResult
in the payout interfaces module, consistent with the existing
codebase convention.
Revert the buffer (50%) and floor (80k) introduced earlier;
the existing 20% buffer is sufficient in practice. Recovery from
the rare OOG case is now handled exclusively by the
checkPayoutCompletionData rollback/retry path.
@TaprootFreak TaprootFreak requested a review from davidleomay May 12, 2026 12:48
@TaprootFreak TaprootFreak marked this pull request as ready for review May 12, 2026 12:49
Comment thread src/subdomains/supporting/payout/strategies/payout/impl/base/evm.strategy.ts Outdated
…dPayout

Address review feedback: route both recoverable cases (mined-OOG and
mempool-expired) through canRetryFailedPayout, with the OOG-specific
nonce rollback handled in the caller. Non-OOG reverts continue to
short-circuit to designatePayout.
…er order

Check status.isOutOfGas explicitly inside the retry branch so the
intent is self-documenting and robust against future changes to
canRetryFailedPayout that could let non-OOG failures through.
@TaprootFreak TaprootFreak merged commit 9390b34 into develop May 12, 2026
7 checks passed
@TaprootFreak TaprootFreak deleted the fix/evm-payout-oog-recovery branch May 12, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants