Skip to content

On-chain first payment race: VM purged as unpaid if confirmation takes longer than the 1h payment expiry #194

Description

@v0l

Problem

A brand-new VM paid on-chain can be hard-purged as unpaid while the customer's payment is sitting in the mempool, if confirmation takes longer than the payment expiry window.

The race

Two 1-hour windows coincide:

  1. On-chain payments are created with expires = now + 3600s (ONCHAIN_EXPIRE, lnvps_api/src/subscription/mod.rs).
  2. The worker purges never-paid VMs once now - sub.created > 1 hour and !sub.is_setup (lnvps_api/src/worker.rs, vm_old_enough_to_delete). This is a hard purge: delete_vm(vm.id, true).
  3. The only thing that defers deletion is list_pending_vm_subscription_payments, whose SQL requires is_paid = 0 AND expires > NOW() — an expired on-chain payment no longer counts as pending.

Timeline: VM + on-chain payment created at t0; customer broadcasts at t0+50m; block takes 30 min. At t0+1h the payment is expired, the sub is not set up, the pending-payments guard is empty → VM purged. At t0+80m the deposit confirms → handle_confirmed finds the VM deleted → funds are not credited, admins are notified, and the customer must go through support.

Why Detected doesn't save it

OnChainPaymentHandler::handle_detected (0-conf mempool sighting, lnvps_api/src/payments/onchain.rs) re-prices the payment and stores the outpoint in external_id, but:

  • does not extend payment.expires
  • does not set is_paid or sub.is_setup

So a deposit that is already visible in the mempool provides zero protection against the unpaid-VM purge. (It even has an explicit "VM deleted → ignore" early-return.)

If the watcher was down during broadcast, Detected is never seen at all and the first event is Confirmed — potentially hours later.

Suggested fix

Options (can combine):

  1. Extend expiry on detection — in handle_detected, push payment.expires forward (e.g. now + 24h or until confirmation) when a mempool deposit is matched to the pending payment. The pending-payments guard then holds the VM automatically.
  2. Guard the purge against detected deposits — before purging, also treat an on-chain payment with a set external_id (deposit seen, not yet confirmed) as pending regardless of expires.
  3. Consider whether ONCHAIN_EXPIRE = 3600 is realistic at all for a first payment quote — a longer window (or decoupling "rate quote validity" from "payment liveness") avoids the coincidence with the 1h unpaid-VM purge. Note: since issue On-chain payments #109 the quote is discarded and re-priced at detection anyway, so the 1h expiry no longer protects the rate — it only creates this race.

Impact

  • Customer pays, VM is destroyed anyway, funds arrive to a deleted VM and must be manually resolved via support.
  • Hard purge means the VM row is gone entirely (delete_vm(..., true)), so recovery = re-provisioning + manual crediting.

Acceptance criteria

  • A deposit detected in the mempool for a pending first payment prevents the unpaid-VM purge until it confirms or is abandoned.
  • A confirmation arriving after the original 1h quote expiry still settles the payment and sets up the VM (given the VM was held).
  • Regression test covering: VM created → on-chain payment created → Detected at t < 1h → worker purge cycle at t > 1h keeps the VM → Confirmed settles it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpaymentsPayment/invoice logic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions