listunspent is really slow

[RHavar]

Even with a small (~50) amount of inputs, calling bitcoin-cli listunspent is rather slow taking between 7 and 15 seconds on a high powered machine with lots of memory. This also makes a bunch of other things rather painful (for instance sending funds).

No doubt the issue is exacerbated by my large (~1GB) wallet file containing about ~520k transactions; but I assume it means there's some indexes or something missing.

[jonasschnelli]

The current implementation is not optimised for large "enterprise" size wallets. I guess listunspent mainly suffers from the fact that it goes over the complete mapWallet (in memory map of all transactions), in your case ~520k transactions.

There is a little optimisation in master that hasn't been deployed (will be in 0.15):
#8952

[laanwj]

The current process for finding unspents is inefficient. These PRs aim or aimed at improving listunspent performance, I'd recommend investigating them:

• Add index to wallet UTXO Add index to wallet UTXO #7823 (closed)
• Cache vout IsMine() value on CWallet::AvailableCoins() Cache vout IsMine() value on CWallet::AvailableCoins() #9939

[maflcko]

Related: #7475

[gmaxwell]

Even with a small (~50) amount of inputs, calling bitcoin-cli listunspent is rather slow taking between 7 and 15 seconds on a high powered machine with lots of memory.

To be clear, this is a wallet with tens of thousands of transactions in it-- right? Listunspent iterates over all the transactions, not just the ones with unspent outputs.

[RHavar]

@gmaxwell Sorry, I meant a small amount of unspent outputs. (as in, the listunspent return result is quite small) but yeah the wallet itself has a lot of transactions

[promag]

Indeed for large wallets the current implementation is slow and @joaopaulofonseca tried to improve parts of the algorithm. In the past I've submitted #7649 which improved a bit. It is hard to improve taking the current wallet implementation.

If you must create a lot of transactions then I can suggest:

• cache externally: you periodically retrieve unspents and then try to first spend those when creating new transactions. With that approach you have to deal with change output and fee calculation;
• create a new "send wallet" when the current has more than N transactions. Just transfer the balance to an address of the new wallet.

[maflcko]

The feature request didn't seem to attract much attention in the past. Also, the issue seems not important enough right now to keep it sitting around idle in the list of open issues.

Closing due to lack of interest. Pull requests with improvements are always welcome.