wallet: cache paid-invoice keys to fix GUI freeze after broadcast

[sashazykov]

Fix for a long-standing GUI freeze when broadcasting a transaction from a wallet with many already-paid outgoing invoices that share output scriptpubkeys.

This issue has bothered me personally for nearly ten years — I originally reported it back in 2018 in #4183. The wallet I use most has accumulated 500+ invoices over the years (typical for repeat payments to the same set of addresses), and every time I broadcast a new transaction the GUI would lock up for several seconds — up to a minute on bigger wallets. The workaround I'd been using is wallet.clear_invoices(); wallet.save_db() in the console, but that throws away history I'd rather keep.

Symptom

After clicking Send, the entire Qt GUI becomes unresponsive for several seconds (tens of seconds on larger wallets). Scales with the number of stored invoices, not with the size of the new transaction.

Root cause

_invoices_from_scriptpubkey_map maps scriptpubkey -> set[invoice_key]. When many invoices share an output scriptpubkey (very common in real wallets — repeat payments to the same destination), this set is large. On broadcast:

1. set_broadcasting(tx, PR_BROADCASTING) iterates every touched invoice and calls get_invoice_status on each.
2. _update_onchain_invoice_paid_detection runs the same loop again after the tx is added to the address synchronizer.
3. set_broadcasting(tx, PR_BROADCAST) runs it a third time on success.

Each get_invoice_status on an onchain invoice calls _is_onchain_invoice_paid, which scans every output scripthash against _prevouts_by_scripthash and does a get_tx_height per prevout. With shared inputs/outputs and many past payments, both inner dimensions blow up together. All of this runs on the GUI thread.

Fix

Two commits:

wallet: cache paid-invoice keys to avoid prevout rescan
Adds _paid_invoice_keys: Set[str] to Abstract_Wallet, an in-memory cache of outgoing invoice ids known to be PR_PAID, maintained incrementally:

• populated at wallet load by _prepare_onchain_invoice_paid_detection
• updated in _update_onchain_invoice_paid_detection (discard then recompute, so reorgs still demote PAID->UNPAID)
• updated in save_invoice, delete_invoice, clear_invoices, clear_history
• new on_event_invoice_status listener keeps it in sync with LN-driven transitions from LNWallet.set_invoice_status

Both save_invoice and _update_onchain_invoice_paid_detection reuse the result of the existing (is_paid, conf) computation instead of triggering a second prevout scan via get_invoice_status.

get_invoice_status short-circuits to PR_PAID on cache hit. set_broadcasting skips already-paid invoices entirely — broadcasting_status has no effect on a paid invoice and the callback churn is what made the GUI freeze.

qt/send_tab: drop redundant invoice_list.update() after broadcast
The explicit full rebuild after a successful broadcast is unnecessary: the set_broadcasting(PR_BROADCAST) call immediately after fires the invoice_status callback for any touched invoices, and main_window.on_event_invoice_status updates the affected rows incrementally. Tx ingestion also sets need_update, which causes update_tabs() to refresh.

Testing

Unit tests: added TestOutgoingInvoicesPaidCache (9 tests) covering: empty/populated cache state, invoice-paid event flow, delete/clear cleanup, the short-circuit in get_invoice_status, set_broadcasting skipping cached-paid invoices, the shared-scriptpubkey regression scenario, cache population at wallet load, cache demotion on reorg (verified tx unverified), and cache reset on clear_history.

$ python -m unittest tests.test_invoices tests.test_wallet tests.test_wallet_vertical
Ran 128 tests in 1.65s
OK

Also ran tests.test_lnpeer and tests.test_lnwallet — 143/143 pass.

Real-world testing: I ran a patched build on macOS against my actual long-lived wallet — the one with the 500+ invoices that motivated the fix. The freeze is gone; sending a transaction is now instant.

Notes for reviewers

• The fix is fully invariant-preserving: every status transition path (including reorg-driven PAID->UNPAID via on_event_adb_removed_verified_tx, and history wipe via clear_history) still flows through code that discards the cache entry before any subsequent read. Both transitions have explicit regression tests.
• Thread-safety: _paid_invoice_keys is only ever touched with single set operations (add / discard / in / clear), no iteration, which is safe under the GIL.
• The on_event_invoice_status listener firing on the wallet's own callbacks is idempotent and intentional (defense in depth for any future callsite that fires invoice_status without going through the existing paths).
• Wallet load itself is incidentally a bit faster: _update_onchain_invoice_paid_detection now derives status from the already-computed (is_paid, conf_needed) tuple instead of calling get_invoice_status, which previously re-ran is_onchain_invoice_paid — halving the per-invoice prevout work in _prepare_onchain_invoice_paid_detection.