cost too many fees?

[JokerCatz]

Hi , I send a transaction , but it cost 5,209.456 sat/B ...... & used txin all over 50 confirms

bitcoin-cli sendtoaddress 1846VsYbYJmqLteuMLdZAYxoVDsjWZyMRz 20

https://blockchain.info/tx/37f98737fe28f1f29840926edeb9cb3a85eb9496070048eb98f2523d612c3cfb

and my getinfo

{
  "version": 120100,
  "protocolversion": 70012,
  "walletversion": 60000,
  "balance": SKIP,
  "blocks": 462806,
  "timeoffset": 0,
  "connections": 8,
  "proxy": "",
  "difficulty": 520808749422.1398,
  "testnet": false,
  "keypoololdest": 1485144781,
  "keypoolsize": 301,
  "paytxfee": 0.00000000,
  "relayfee": 0.00001000,
  "errors": "Warning: Unknown block versions being mined! It's possible unknown rules are in effect"
}

& my config

upnp=0
server=1
rpcuser=SKIP
rpcpassword=SKIP
rpcthreads=8
rpckeepalive=false
rpctimeout=30
rpcallowip=127.0.0.1
rpcport=8332
keypool=300
mintxfee=0.00000001
checkblocks=16

did it just version too old & need to upgrade?

[jonasschnelli]

Please use something like http://bitcoin.stackexchange.com for future user questions. This is the development issue tracker.

It looks like that your fee estimation spits out a relatively high value.
Try to upgrade to 0.14.0.
Maybe play with txconfirmtarget=<your desired target in blocks>.

[NicolasDorier]

I think this question have its place here, it might have been a bug.

[morcos]

You should definitely upgrade to 0.14.0, but it is still possible although rare for such a result to happen.

Do you know what your transaction confirm target was set to? The default in 0.12 was changed to 2, but if you are running the GUI, then the GUI setting from a previous version may have persisted.

This kind of result is much more likely with a target of 1 than with a target of 2.
Personally I would recommend increasing your transaction confirm target to at least 6 blocks.

[JokerCatz]

I use ubuntu server only , and install from tarball binary(not compiler by my self) , and I send 5 transactions same time at 3 node(different VM) , same config same amount same version , nodeA x 3 , nodeB x 1 , nodeC x 1 , and it only happen in nodeA

1.fail 73b8a20980eca8706a083e0bc511af8a8a93e408228411fafe5f05487a4d191d 2,100.84_ sat/B
2.okay f343524000eaedcbc61d68367facea6d268941ba8d464075048c2a2bf056f649   188.295 sat/B
3.fail 37f98737fe28f1f29840926edeb9cb3a85eb9496070048eb98f2523d612c3cfb 5,209.456 sat/B

hmm ...... anyway I'll upgrade my node , and add confirm target(maybe 4) , and maybe rebuild the nodeA

& thanx @morcos recommand : ) , first post is my all config file , so I think it's default value ...

[morcos]

Those are really very high values, so I am a bit curious how that could have happened.
Had your node not been running very long?

But this is a good reminder that we should record more debugging information by default in the log when a transaction automatically uses fee estimation.

[laanwj]

But this is a good reminder that we should record more debugging information by default in the log when a transaction automatically uses fee estimation.

Having fee estimation more auditable would be a great idea.

[JokerCatz]

hmm @morcos it's my production node , it run over 2 month ... but I write my code to make & send rawtransaction to clear dust (very low fee) , but this time is some emergency things must immediately to send , so I send it from terminal

so it possible the fee calculate weight include another older transaction like manual to send rawtransaction ? even no relation with another non-used vouts ? & just I say , picked vouts all over 50 comfirms

[morcos]

@laanwj just closed #4082 (because it is now fixed) and I don't know why I didn't think of that before, but it seems possible that was the cause of this problem and not an error in fee estimation. Especially given that there was a normal transaction in between.

[dabura667]

bitcoin-cli sendtoaddress 1846VsYbYJmqLteuMLdZAYxoVDsjWZyMRz 20

I would like to point out:

1. the only output is exactly 20 BTC (the amount entered in the command)
2. 3 of the lowest value inputs could have been omitted and the transaction would have still been in the 3000 sat/byte range... However, those low value inputs were not low enough to be considered unusable imo.

[instagibbs]

just to note: A user on IRC last night reported extremely high fees(2,000+ sat/byte) a couple times using sendmany on 0.14.1 even though settxfee was set.

[maflcko]

Did they set subtract_fee_from_amount=True? That is still an open issue: https://github.com/bitcoin/bitcoin/blob/80c3a734298e824f9321c4efdd446086a3baad89/src/wallet/wallet.cpp#L2627

…

On Sun, Apr 30, 2017 at 3:58 PM, Gregory Sanders ***@***.***> wrote: just to note: A user on IRC last night reported extremely high fees(2,000+ sat/byte) a couple times using sendmany on 0.14.1 even though settxfee was set. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#10247 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGGmv8l7hG66vIpifetrAsuNdM32kxtlks5r1JOcgaJpZM4ND3PA> .

[cncr04s]

@MarcoFalke I did not set subtract_fee_from_amount. I'm just simply using sendmany to multiple addresses.
It created a few transactions with 2000+ sat/byte, used settxfee specifically and txconfirmtarget=4. It usually creates transaction correctly with change and the correct fee, so in my case the same behavior as the OP.

It appears to me as it ate the rest of the change for the fee in all cases. My inputs vary widely in size, there are usually a few inputs less than 0.05 for it to choose from when building it, usually from existing change. 80% of the time it usually comes up with an input set that is within ~0.01 of the output sum plus fee.

Additionally, occasionally sendtoaddress would send with a fee a bit higher than I expected, decided to try sendmany to reduce fees and combine individual sends into a single transaction.

Using this in a production environment, not expecting bitcoin core to have issues like this.

[instagibbs]

@MarcoFalke It may be the line two lines down:

bitcoin/src/wallet/wallet.cpp

Line 2630 in 80c3a73

// TODO: The case where there is no change output remains

Since he has no change in his transactions with such a high fee.

[RHavar]

@JokerCatz If you run bitcoin-cli listunspent | grep amount | wc -l what do you get? This used to be a really common issue when you had lots of unspent, but I haven't noticed it running the latest version

[JokerCatz]

I have 4 full node : [777 , 454 , 390 , 225]

[morcos]

As mentioned on IRC https://botbot.me/freenode/bitcoin-core-dev/2017-05-04/?msg=85156140&page=2 , I think its still possible for the old coin selection problem to rear it's head. I'm not sure exactly how rare it is, but should be significantly less common than it used to be.
#10333 could be one way to avoid this, i haven't reviewed it yet, but I'm hoping to get to a more complete rewrite of the coin selection algorithm.

[RHavar]

@JokerCatz You have a lot of unspent on all your nodes, you really should upgrade them all ASAP. In total I've lost over a bitcoin in total due to the bug that was fixed (or at least, heavily mitigated), it'll probably be affecting you often when you send money

[instagibbs]

Since this issue appears to be the "no change" situation, it's likely not mitigated yet.

[JokerCatz]

@RHavar my production normally use rawtransaction send and clear dust , so it not problem & thanx , I'll upgrade my all node next week , just it look like bug & report it

[cncr04s]

Is there any update on this issue?

[instagibbs]

@cncr04s #10333 should fix this particular case, and is targeted to 0.15. We'll see if it can get enough review.

[ysangkok]

@JokerCatz Did you test with a 0.15 release candidate? It would be interesting to see if it fixes your problem.

[JokerCatz]

Hi , I'm so sorry , my project is fixed , use rawtransaction to send transaction & calculate fees... so I think I can't test this issue ...

[morcos]

I think this can be closed

[cncr04s]

I am also having this problem, I may be able to test the actual release though. I havent kept up on when that would be. I run live environment, so may be a while till I can get an answer. I do however see many other incoming transactions having the same fee overpayment issue from time to time.

…

On Sep 7, 2017 1:00 PM, "Alex Morcos" ***@***.***> wrote: I think this can be closed — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10247 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABaLWkuM5WJbnQc3XTG05BDI67n0KbUDks5sgC5TgaJpZM4ND3PA> .

[instagibbs]

@cncr04s 0.15/master should catch any of these types of vast overpayment errors.

[morcos]

@cncr04s If you are running 0.14, then it should be very relatively rare to have overpayment issues, but they can still exist when you have a transaction without change. That edge case should be fixed in 0.15. If you have this problem on 0.14 node on a transaction with change, please provide us with us much information as you can.

[Safari77]

at least 0.15.1 is not doing overpayment, with txconfirmtarget=6 it uses 20 sat/byte, and tx still unconfirmed after two days.
😑

Fee Calculation: Fee:4520 Bytes:226 Needed:4520 Tgt:0 (requested 6) Reason:"Fallback fee" Decay 0.00000: Estimation: (-1 - -1) -nan% 0.0/(0.0 0 mem 0.0 out) Fail: (-1 - -1) -nan% 0.0/(0.0 0 mem 0.0 out)

[JokerCatz]

@Safari77 did your mempool size is too small ? & is sync finished ? or offline signature ? , I tried use estimatesmartfee will use mempool to fees , and you can use estimatefee / estimatesmartfee to calculate it first , but it per kb fees , so you need * 100000000 / 1024 will be per byte satoshi

[morcos]

@Safari77 When it says that the reason is "Fallback Fee" that means fee estimation is not working.
Seems likely that this was a new node and had received less than 4 or 5 blocks since syncing. If using the GUI, there should be a red warning, but using bitcoin-cli we don't currently have any way of warning people that it's not possible to do fee estimation yet. There is really no solution to this other than to get your fee estimate from other sources and manually set the fee.

[Safari77]

Well I definitely was not inspecting debug.log before sending the transaction in GUI after it synced and also didn't have an idea about that 4-5 block "rule"...

So my only option is to quit bitcoin-qt, delete mempool.dat, and start bitcoin-qt again?

[instagibbs]

No do not delete your mempool data. Let your node run for a while and it will fix itself.

…

On Dec 10, 2017 9:58 AM, "Sami Farin" ***@***.***> wrote: Well I definitely was not inspecting debug.log before sending the transaction in GUI *after* it synced and also didn't have an idea about that 4-5 block "rule"... So my only option is to quit bitcoin-qt, delete mempool.dat, and start bitcoin-qt again? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#10247 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFgC02g-zN9LKc-zM3dvHsfSmwMSyP5Lks5s-_GogaJpZM4ND3PA> .

[morcos]

@Safari77 Just to clarify, i was suggesting that if you sent transaction from bitcoin-qt you should have seen something like this and there should have been no need to inspect debug.log:

But @instagibbs is right, you should not need to do anything to "fix" the problem. You should be able to see from bitcoin-qt that the estimates are more reasonable now.

[Safari77]

Yes, it now suggests Transaction Fee 0.0019...
So I guess I missed that warning. Maybe it could ask for extra confirmation if Smart free is not yet initialized in case user is half-asleep.

[morcos]

Yeah that’s not a bad idea
Ping @jonasschnelli

BTW you can use bumpfee if you had RBF enabled and want to pay more fee to get it confirmed faster.

[jonasschnelli]

Thing that could be improved:

a) Don't allow to send transactions when fee estimation is not initialised
b) Show an additional popup warning
c) Same a b) but force BIP125-RBF

I prefere c.

[instagibbs]

I like (c), or (b) at a minimum. (a) is a non-starter, if nothing else because that would make regtest hard to use

[morcos]

I wouldn't force BIP125-RBF, but additional popup warning could strongly recommend it

[cncr04s]

Can we add a API function that estimates the fee. I looked at the code and there is such an internal function, ie it creates the transaction object then sends it to the network. I propose that the estimate fee would just return the created transaction without relaying it to the network. This way, users can decide for them selves if the fee being paid is appropriate for their needs.
Thanks.

[sipa]

@cncr04s We have estimatefee (deprecated), estimatesmartfee, and estimaterawfee for getting feerate estimates. Furthermore, the fundrawtransaction will add inputs and change to a transaction according to the fee estimation, without broadcasting.

[cncr04s]

@sipa I'm not sure that applies to a sendtoaddress request. I want to know how much fee is paid in the sendtoaddress before its actually sent. I don't know what inputs it selects/ the size of the transaction and therefore can't calculate the fee using estimatesmartfee.

[sipa]

@cncr04s You can use fundrawtransaction for that.

[maflcko]

To the best of my knowledge this issue can be closed, as we fixed all known instances of "excessive fee-rate", i.e. accidentally creating a transaction that pays more satoshis per byte in fees than it should (taking into account the target fee rate).

The issue that coin selection will yield a very unfortunate subset of coins, and cause an "excessive overall fee", i.e. a transaction which spends many "uneconomical coins" is tracked in #1643.

The issue that fallbackfee silently results in "insufficient fee-rate or overall fee" is a completely separate issue and should be tracked on a separate ticket. Solutions to improve the fallbackfee situations are proposed in: #11918, #11882, #11892, #9481 (merged in 0.15)

[JokerCatz]

@MarcoFalke okay & why not , many thanx dev team :)