Wallet TX history memory and CPU exhaustion

[braydonf]

Versions
bcoin v1.0.2

Expected

Large history of transactions can be queried in smaller sets sequentially in pages with predictable sets of results in ascending and descending order, by time, and without requiring loading all transactions into memory (and crash and overload the process). Should be able support millions of transactions.

Actual

Some endpoints will return the entire history without options to limit or change the order and can exhaust memory. Other endpoints support querying by time; however, with unpredictable size of results without pagination. The time of the index is based on when the transactions was seen to a node, rather than block time that can be recovered from the chain.

Pull Request / Branch:

• Wallet TX count and time indexing #605
• https://github.com/braydonf/bcoin/commits/wallet-pagination

[bucko13]

probably don't want to update the RPC for anything that's already in core so for RPC support, probably a new command. listtransactionhistory?

[braydonf]

Yeah a new command, listhistory, could work. It could search by timestamp and txid and listing in ascending and descending order.

[tynes]

@braydonf Wondering the status on this and if there is anything that I can do to help

[braydonf]

The PR has status details at #605

[braydonf]

The solution for this is implemented at https://github.com/bcoin-org/bcoin/commits/wallet-pagination and should remain as it is currently until the topic is revisited.