multi: Implement smart fee estimation.

[matheusd]

Initial release of the fee estimator.

Closes #1412

[matheusd]

Sending for initial review and to discuss some last points that might require working before merging.

As a preliminary, I'm sending attached the fee database collected over the last few days in mainnet, and the dump of the estimator state for this same database:

feesdb-mainnet.tar.gz
fees.txt

Results of the estimation:

$ ./dcrctl.sh estimatefee 1
-32603: no bucket with the minimum required success percentage found
$ ./dcrctl.sh estimatefee 2
0.00112764
$ ./dcrctl.sh estimatefee 3
0.00112764
$ ./dcrctl.sh estimatefee 4
0.00087775
$ ./dcrctl.sh estimatefee 6
0.00087775

A few notes for discussion:

• I've hooked into the estimatefee instead of the estimatesmartfee because there are no parameters to the estimator right now (eg: conservative vs economical) and the handler was already there, I just had to hook it. Can rework into a brand new handler if that is preferable.

• Those txs with lower fee are already bringing down the estimates vs the current default wallet fee. Eyeballing from logs, they seem to be all revocations, so unsure whether we'll need to special case them.

• The 0.00108347 bucket is mostly filled with tickets, which due to hard coded size estimates end up paying fees slightly larger than strictly required. Hopefully fixing Improve size/fee estimation for tickets dcrwallet#1272 improves this.

• Due to how tickets are managed in the mempool and the fact that they are sent along with split txs (but are only eligible to be included in the block after the split has been mined and validated), they also severely skew the 1 confirmation bucket. Ticket handling in mempool would have to be seriously tweaked to fix this. Specifically, tickets would need to be tracked as out of the mempool (either as orphans or as some other classification) and could only enter the mempool after their respective split had been mined.

[dnldd]

Nicely done @matheusd, numblocks parameter description for estimatefee will have to be updated if the current rpc setup is maintained. Leaving this here as a reminder.

[matheusd]

Current stats after running in mainnet for ~2 weeks.

Internal estimator state

$ ./dcrctl estimatefee 1
-32603: no bucket with the minimum required success percentage found
$ ./dcrctl estimatefee 2
0.00136971
$ ./dcrctl estimatefee 3
0.00114035
$ ./dcrctl estimatefee 4
0.00114035
$ ./dcrctl estimatefee 5
0.00114035
$ ./dcrctl estimatefee 6
0.00087921
$ ./dcrctl estimatefee 7
0.00087921
$ ./dcrctl estimatefee 16
0.00087921

[davecgh]

The EstimateSmartFee command in dcrjson needs to be updated to make the second parameter optional as well. Currently it is required.

[davecgh]

Seems like this should be using a value passed in from elsewhere (perhaps policy.DefaultMinRelayTxFee), especially since wallet should be using 1e4 now. Otherwise this will certainly get out of date.

[matheusd]

Slight difficulty on this one. I've moved the constant out of the fees package to the server.go file, passing it as an ExtraBucketFee to be inserted.

The main issue with using DefaultMinRelayTxFee is that this is not supposed to be current fee, but rather the one that the majority of the wallets are using (ie, the previous DefaultMinRelayTxFee).

So, one alternative would be to explicitly add a mempool.PreviousDefaultMinRelayTxFee and add comments detailing that it should be bumped once DefaultMinRelayTxFee is also bumped.

Another one, to make this less "magical" would be to remove this constant altogether and add explicit tracking of all order-of-magnitude bucket fees in the range (1e3, 1e4, 1e5, etc).

[matheusd]

Squashed. I removed some unnecessary bumps from go.mod and go.sum, but other than that the commits are identical to the previous ones.