"estimatefee 1" returns fee that seems high

[kaykurokawa]

Currently getting 0.44247787 using RPC command "estimatefee 1" , seems unusually high? "estimatefee 2" returns a more reasonable 0.00444444. Was wondering what other people are getting for this.

Using current latest master branch 0.10

[CohibAA]

Same here. Litecoin Core Daemon version v0.10.2.2

litecoin-cli estimatefee 1
0.44444444

More node stats for comparison.

[kaykurokawa]

I think this is a major problem?
https://www.reddit.com/r/litecoin/comments/3gfeip/litecoind_is_charging_a_010_fee_on_litecoin/

I believe by default transactions will try to use the fee estimation with parameter 1 no (inclusion within a single block)? If so, people are paying a lot in transaction fees.

[CohibAA]

If people configure their client and don't use the default config, they can mitigate this, but it has caused some entities to pay more fees than necessary.

[kaykurokawa]

That's strange that tx fee is being set to 0.1? I checked some previous blocks and there is a fair number of transactions with this exact fee, see for example:

http://explorer.litecoin.net/block/16a4bdb70216b29c0e1d36657605d61976ebd140bc8e76ec79285bac1efdaa13

[CohibAA]

That's not strange at all... it's the default maximum. So, when the default recommend next block amount is used, it get overridden by the default max setting.

litecoind --help

-maxtxfee= Maximum total fees to use in a single wallet transaction, setting too low may abort large transactions (default: 0.10)

Why the algorithm thinks such a high fee is necessary is a question I don't yet have an answer to.

[jyap808]

That default is set in:

init.cpp

strUsage += "  -maxtxfee=<amt>        " + strprintf(_("Maximum total fees to use in a single wallet transaction, setting too low may abort large transactions (default: %s)"), FormatMoney(maxTxFee)) + "\n";

wallet.cpp maxTxFee reference

CAmount maxTxFee = DEFAULT_TRANSACTION_MAXFEE;

wallet.h maxTxFee value

static const CAmount DEFAULT_TRANSACTION_MAXFEE = 0.1 * COIN;

[kaykurokawa]

I see, makes sense. CWallet::GetMinimumFee function is responsible for setting the default max fee I believe?

https://github.com/litecoin-project/litecoin/blob/master-0.10/src/wallet.cpp#L1607

If that's the case, it seems to be pretty clear what's happening. Once you have a majority of nodes believing that you need to attach the max fee in order to meet you fee estimate, than the estimate will always stay there because its essentially a feedback loop. I don't know what caused the estimate to bump up initially, but from what I read of the fee estimation code, it doesn't take a whole lot for that to happen (11 samples per fee estimation bucket)

Seems like an urgent fix is required, or at least warn everyone to use settxfee

[wtogami]

I indexed the last several thousand blocks while doing estimatefee 1 every 50 blocks or so. I got nothing but 0.00444444 from estimatefee 1, and 0.00103626 from estimatefee 4 or higher which is close to the mintxfee and what you should expect while there is no fee pressure.

So ...

[kaykurokawa]

Check the blockchain for transactions with transaction fee 0.1 , you will see that there is a bunch.

Since fee estimation is depedent on your txmempool, I think it makes sense that different nodes will have different estimates. But it does seem weired that the estimates are that off. Currenlty, I'm getting 0.12269938 for estimatefee 1.

Not sure how re indexing has an impact on the fee estimation?

[wtogami]

When estimatefee 1 is giving up large numbers, what does estimatefee 4 or 6 give you? 1 by design will give you numbers far in excess of what is actually needed, and most people probably don't want to use 1 on a network where fee pressure is not a concern.

[kaykurokawa]

What my current full node is getting, everything below 1 looks reasonable so I'm wondering if this is some bug in the fee estimation code. Also if "most people probably don't want to use 1" than, maybe it shouldn't be the default option? That is nTxConfirmTarget in wallet.cpp maybe should not be set to 1?

1:0.26809651

2:0.00444444

3:0.00299401

4:0.00205338

5:0.00137803

6:0.00116279

7:0.00113765

8:0.00113765

9:0.00113765

10:0.00113765

11:0.00113765

12:0.00113765

13:0.00113765

14:0.00113765

15:0.00113765

16:0.00113765

17:0.00113765

18:0.00113765

19:0.00113765

20:0.00113765

21:0.00113765

22:0.00113765

23:0.00113765

24:0.00113765

[losh11]

Yeah, some company who does do payouts uses estimate fee 1 which charges back the customers quite some money.

[wtogami]

Where is estimatefee 1 default? If that is default somewhere that should probably be changed to 2, 3, or maybe 4.

Keep in mind that 4 is equal in time to Bitcoin's 1.

There may also be a legitimate bug making the estimate of 1 wildly off, but I am not able to reproduce that here. @thrasher- can you look into this?

[kaykurokawa]

So after poking around, the way the fee estimation works, when you say "estimatefee X", it means "Find the median transaction fee from transactions in my mem pool that confirmed within X blocks or faster"

So naturally, when X=1, it uses less samples than when X>1, and is prone to being affected by few transactions with high tx fees. I believe it stores 100 transactions in the mempool for each target confirmation bucket, so in theory, if you burst 50 transactions with high tx fees, you can quickly push the tx fee high for a lot of nodes in the network. And once other nodes in the network start believing in its own estimation and using it, the problem worsens.

I think as mentioned, nTxConfirmTarget (see below) should be set higher.

https://github.com/litecoin-project/litecoin/blob/master-0.10/src/wallet.cpp#L30

We should urge users to set -txconfirmtarget to greater than 1 as well in the mean time.

NOTE: Bitcoin currently uses 2 as the default tx confirm target

[thrasher-]

Thanks for reporting this, Litecoin-Qt users will be fine from this because the normal block target starts at 25 and then ranges to 1 (fast). However, people who use litecoin-cli or the RPC sendtoaddress command will use the nTxConfirmTarget (currently 1) setting, thus using a higher fee.

This has been bumped up to 2 for Bitcoin 0.11 and was 1 in 0.10. I'm currently testing your patch and will merge if its fine.

[thrasher-]

Closed as PR has been merged.