Treat dust outputs as non-standard, un-hardcode TX_FEE constants #2577

Merged
merged 3 commits into from May 4, 2013

Conversation

Projects
None yet
@gavinandresen
Contributor

gavinandresen commented Apr 26, 2013

This is a quick, safe fix for transaction fee handling. A riskier, more complicated fix still needs to happen, but will take more time to code/test/etc.

Two motivations for this pull:

First, to discourage people from bloating users' wallets and the UTXO set with tiny outputs.

This pull defines 'uneconomic dust' as 54.3 uBTC (5430 satoshis, about $0.007 at current prices), and treats any transaction with outputs less than 5430 satoshis as non-standard (won't be relayed, won't be mined). 5430 satoshis is derived from the cost (in fees) to spend a TxOut/TxIn. See https://people.xiph.org/~greg/txouts2.png for proportion of recent outputs this will (eventually) affect.

Second, we have no idea what will happen to Bitcoin prices or transaction volume / competition for space in blocks. So this patch lets users and miners specify the minimum transaction fees at startup, using the -mintxfee / -mintxrelayfee options (which I'm intentionally leaving undocumented for now). The dust/nonstandard test is based on the -mintxrelayfee.

Qt and RPC both now tell the user why CreateTransaction fails, if it fails; Qt error reporting is a little wonky (try to send one satoshi, and you get two modal dialog boxes, one after the other; I don't care enough to try to fix that).

Note: I spent several hours trying, and failing, to create unit tests for this patch; CWallet::fFileBacked is ignored by several of the wallet routines used by CWallet::CreateNewTransaction. In the end, I decided thatrefactoring CWallet to support unit testing would be much more extensive and riskier than these changes.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 26, 2013

Member

We'd also want to change #2576

Member

laanwj commented Apr 26, 2013

We'd also want to change #2576

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik Apr 26, 2013

Contributor

ACK

Contributor

jgarzik commented Apr 26, 2013

ACK

@gmaxwell

This comment has been minimized.

Show comment
Hide comment
@gmaxwell

gmaxwell Apr 26, 2013

Member

Gavin, Can you make IsDust() return true for zero no matter what value MIN_RELAY_TX_FEE is or just make the ==0 test separate? I don't want to open up zero value floods for people who decide that they want to lower mintx/mintxrelay and find that 0 is easier to type than 0.0000001.

Member

gmaxwell commented Apr 26, 2013

Gavin, Can you make IsDust() return true for zero no matter what value MIN_RELAY_TX_FEE is or just make the ==0 test separate? I don't want to open up zero value floods for people who decide that they want to lower mintx/mintxrelay and find that 0 is easier to type than 0.0000001.

@gavinandresen

This comment has been minimized.

Show comment
Hide comment
@gavinandresen

gavinandresen Apr 26, 2013

Contributor

It won't open up flooding if the rest of the network refuses to relay or mine those transactions. And I hate making code that I'm planning on replacing soon more complicated.

Contributor

gavinandresen commented Apr 26, 2013

It won't open up flooding if the rest of the network refuses to relay or mine those transactions. And I hate making code that I'm planning on replacing soon more complicated.

@Diapolo

View changes

src/qt/walletmodel.cpp
@@ -189,6 +190,7 @@ bool WalletModel::validateAddress(const QString &address)
{
return SendCoinsReturn(AmountWithFeeExceedsBalance, nFeeRequired);
}
+ emit message(tr("Send Coins"), QString::fromStdString(strFailReason), CClientUIInterface::MODAL);

This comment has been minimized.

@Diapolo

Diapolo Apr 26, 2013

Gavin, if you want this to be an error please use CClientUIInterface::MSG_ERROR or CClientUIInterface::MSG_WARNING for a warning :).

@Diapolo

Diapolo Apr 26, 2013

Gavin, if you want this to be an error please use CClientUIInterface::MSG_ERROR or CClientUIInterface::MSG_WARNING for a warning :).

+ // a transaction spammer can cheaply fill blocks using
+ // 1-satoshi-fee transactions. It should be set above the real
+ // cost to you of processing a transaction.
+ if (mapArgs.count("-mintxfee"))

This comment has been minimized.

@Diapolo

Diapolo Apr 26, 2013

These 2 switches are not mentioned in our init.cpp help-message, is this intended?

@Diapolo

Diapolo Apr 26, 2013

These 2 switches are not mentioned in our init.cpp help-message, is this intended?

This comment has been minimized.

@gmaxwell

gmaxwell Apr 26, 2013

Member

(which I'm intentionally leaving undocumented for now)

Yes.

@gmaxwell

gmaxwell Apr 26, 2013

Member

(which I'm intentionally leaving undocumented for now)

Yes.

@gavinandresen

This comment has been minimized.

Show comment
Hide comment
@gavinandresen

gavinandresen Apr 26, 2013

Contributor

Rebased to make the qt an Error as suggested by @Diapolo, and tell the user they screwed up if they give bad -mintxfee/minrelaytxfee values.

Don't know why github is confused about the commits...

Contributor

gavinandresen commented Apr 26, 2013

Rebased to make the qt an Error as suggested by @Diapolo, and tell the user they screwed up if they give bad -mintxfee/minrelaytxfee values.

Don't know why github is confused about the commits...

@gmaxwell

This comment has been minimized.

Show comment
Hide comment
@gmaxwell

gmaxwell Apr 26, 2013

Member

ACK on approach now. Will test.

Member

gmaxwell commented Apr 26, 2013

ACK on approach now. Will test.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 26, 2013

Contributor

You still need to change CWallet::CreateTransaction() so it doesn't create dust change txouts; see petertodd@0df65da#L1L1192

EDIT: no, actually that's specific to the way my patch works, because in it dust could be defined as greater than nMinFee, so you are correct.

Contributor

petertodd commented Apr 26, 2013

You still need to change CWallet::CreateTransaction() so it doesn't create dust change txouts; see petertodd@0df65da#L1L1192

EDIT: no, actually that's specific to the way my patch works, because in it dust could be defined as greater than nMinFee, so you are correct.

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Apr 26, 2013

Member

Will this prevent spending from transactions with dust outputs as well (since we now check that inputs are standard)?

Member

luke-jr commented Apr 26, 2013

Will this prevent spending from transactions with dust outputs as well (since we now check that inputs are standard)?

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 26, 2013

Contributor

No.

See CTransaction::AreInputsStandard()

Contributor

petertodd commented Apr 26, 2013

No.

See CTransaction::AreInputsStandard()

@gavinandresen

This comment has been minimized.

Show comment
Hide comment
@gavinandresen

gavinandresen Apr 27, 2013

Contributor

@petertodd : I added belt-and-suspenders code, since we don't have good CreateTransaction unit tests.
@sipa: why not indeed.

I'll squash the two commits before final pull.

Contributor

gavinandresen commented Apr 27, 2013

@petertodd : I added belt-and-suspenders code, since we don't have good CreateTransaction unit tests.
@sipa: why not indeed.

I'll squash the two commits before final pull.

@BitcoinPullTester

This comment has been minimized.

Show comment
Hide comment
@BitcoinPullTester

BitcoinPullTester Apr 27, 2013

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/9ac9a459c3fcab63e70d3780d730744cc55ba76f for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/9ac9a459c3fcab63e70d3780d730744cc55ba76f for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

@Diapolo

View changes

src/qt/walletmodel.cpp
@@ -189,6 +190,8 @@ bool WalletModel::validateAddress(const QString &address)
{
return SendCoinsReturn(AmountWithFeeExceedsBalance, nFeeRequired);
}
+ emit message(tr("Send Coins"), QString::fromStdString(strFailReason),
+ CClientUIInterface::MODAL|CClientUIInterface::MSG_ERROR);

This comment has been minimized.

@Diapolo

Diapolo Apr 27, 2013

@gavinandresen CClientUIInterface::MSG_ERROR includes CClientUIInterface::MODAL, that's why I mentioned it the first time :).

@Diapolo

Diapolo Apr 27, 2013

@gavinandresen CClientUIInterface::MSG_ERROR includes CClientUIInterface::MODAL, that's why I mentioned it the first time :).

@mikehearn

This comment has been minimized.

Show comment
Hide comment
@mikehearn

mikehearn Apr 28, 2013

Contributor

What is the longer term plan for this? 0.007 dollars is not very small. It only takes one doubling of the price to mean you can no longer send a payment of 1 cent, which is pathetic, even physical cash can handle that. And in the past we've seen 10x increases in price happen extremely fast. 1 cent isn't even a micropayment, that's just "payment".

Whatever the ultimate plan is, it doesn't seem like this intermediate state is going to be very helpful. It'll just result in the same problem we have today, hard-coded magic numbers that become inappropriate much faster than they can be changed. Do you really see node operators changing the command line flags?

I really think we should scrap the word "dust". It implies some kind of constant valuation, which Bitcoin doesn't have. What looks pointless today can look useful tomorrow.

Contributor

mikehearn commented Apr 28, 2013

What is the longer term plan for this? 0.007 dollars is not very small. It only takes one doubling of the price to mean you can no longer send a payment of 1 cent, which is pathetic, even physical cash can handle that. And in the past we've seen 10x increases in price happen extremely fast. 1 cent isn't even a micropayment, that's just "payment".

Whatever the ultimate plan is, it doesn't seem like this intermediate state is going to be very helpful. It'll just result in the same problem we have today, hard-coded magic numbers that become inappropriate much faster than they can be changed. Do you really see node operators changing the command line flags?

I really think we should scrap the word "dust". It implies some kind of constant valuation, which Bitcoin doesn't have. What looks pointless today can look useful tomorrow.

@mikehearn

This comment has been minimized.

Show comment
Hide comment
@mikehearn

mikehearn Apr 28, 2013

Contributor

OK, to clarify, I agree that this change is an improvement over the old state in that it at least reduces the quantity of magic numbers and makes them derived from a single value. But it'd be nice to know the roadmap here.

Contributor

mikehearn commented Apr 28, 2013

OK, to clarify, I agree that this change is an improvement over the old state in that it at least reduces the quantity of magic numbers and makes them derived from a single value. But it'd be nice to know the roadmap here.

@johndillon

This comment has been minimized.

Show comment
Hide comment
@johndillon

johndillon Apr 28, 2013

Ack

Yesterday the fingerprints of every key in the PGP strong set as well as every key on bitcoin-otc were timestamped permanently in the blockchain for a cost of only $150 or 0.3 cents per timestamp. (I also paid 1.5BTC to the person who did it for me for their time)

50,000 new entries in the UTXO set that all validating nodes will forever have to have a copy of unless Bitcoin develops centralized blacklisting schemes. (yes I know you can use the data to prove it doesn't belong in the blockchain, they are SHA1 hashes so have fun if SHA1 is broken)

According to my PGP keyserver data dump there are 3.2 million PGP keys floating around on keyservers. Timestamping every last one of them would just cost about $32,000 at a penny each for fees and vanished bitcoins, and add 120MB to the txout set. $32k may buy a lot of harddrives, but it doesn't buy enough harddrives to really pay the cost of storing those transactions forever.

Notice how I didn't say "attacker". Timestamping is just one of many reasons why demand for transactions is unlimited.

The developers are absolutely right to price such junk out of the blockchain but aren't going far enough.

Ack

Yesterday the fingerprints of every key in the PGP strong set as well as every key on bitcoin-otc were timestamped permanently in the blockchain for a cost of only $150 or 0.3 cents per timestamp. (I also paid 1.5BTC to the person who did it for me for their time)

50,000 new entries in the UTXO set that all validating nodes will forever have to have a copy of unless Bitcoin develops centralized blacklisting schemes. (yes I know you can use the data to prove it doesn't belong in the blockchain, they are SHA1 hashes so have fun if SHA1 is broken)

According to my PGP keyserver data dump there are 3.2 million PGP keys floating around on keyservers. Timestamping every last one of them would just cost about $32,000 at a penny each for fees and vanished bitcoins, and add 120MB to the txout set. $32k may buy a lot of harddrives, but it doesn't buy enough harddrives to really pay the cost of storing those transactions forever.

Notice how I didn't say "attacker". Timestamping is just one of many reasons why demand for transactions is unlimited.

The developers are absolutely right to price such junk out of the blockchain but aren't going far enough.

@MeniRosenfeld

This comment has been minimized.

Show comment
Hide comment
@MeniRosenfeld

MeniRosenfeld Apr 28, 2013

This will make it impossible to use Bitcoin for colored coins, and some other applications which have information content beyond the mere transfer of value.

I am of the opinion that people should be able to do what they want in the transaction, as long as they pay for the resources consumed.

This will make it impossible to use Bitcoin for colored coins, and some other applications which have information content beyond the mere transfer of value.

I am of the opinion that people should be able to do what they want in the transaction, as long as they pay for the resources consumed.

@gavinandresen

This comment has been minimized.

Show comment
Hide comment
@gavinandresen

gavinandresen Apr 28, 2013

Contributor

Road map:

Re-implement the memory pool, and keep statistics on transactions that enter and then leave by being included in a block. Protect CTransaction::nMinFee/nMinRelayFee with a mutex, and modify those by that memory pool code.

So IsDust() will be based on what is actually being accepted into blocks, and will adjust appropriately.

Do the same for free transactions (estimate priority needed to get included in block).

Modify the RPC code to either send for free (if priority is high enough, based on estimate) or send with reasonable fee (based on fee estimate).

And modify GUI code to either just send for free or recommend sending with a fee.

Contributor

gavinandresen commented Apr 28, 2013

Road map:

Re-implement the memory pool, and keep statistics on transactions that enter and then leave by being included in a block. Protect CTransaction::nMinFee/nMinRelayFee with a mutex, and modify those by that memory pool code.

So IsDust() will be based on what is actually being accepted into blocks, and will adjust appropriately.

Do the same for free transactions (estimate priority needed to get included in block).

Modify the RPC code to either send for free (if priority is high enough, based on estimate) or send with reasonable fee (based on fee estimate).

And modify GUI code to either just send for free or recommend sending with a fee.

@johndillon

This comment has been minimized.

Show comment
Hide comment
@johndillon

johndillon Apr 28, 2013

@gavinandresen Nice technical details. But how expensive do you expect a transaction to be? That is the real issue and that is the one that drives scalability. What are you going to tell the people doing tiny transactions like tipjars, just leave Bitcoin?

@gavinandresen Nice technical details. But how expensive do you expect a transaction to be? That is the real issue and that is the one that drives scalability. What are you going to tell the people doing tiny transactions like tipjars, just leave Bitcoin?

@gavinandresen

This comment has been minimized.

Show comment
Hide comment
@gavinandresen

gavinandresen Apr 28, 2013

Contributor

@johndillon : Bitcoin is not appropriate for transactions less than a penny or three.

If Moore's Law continues to hold, then one day it might be.

That is all out of scope for this pull request; if we do nothing, then we are stuck with MIN_RELAY_TX_FEE=0.0005 BTC.

Contributor

gavinandresen commented Apr 28, 2013

@johndillon : Bitcoin is not appropriate for transactions less than a penny or three.

If Moore's Law continues to hold, then one day it might be.

That is all out of scope for this pull request; if we do nothing, then we are stuck with MIN_RELAY_TX_FEE=0.0005 BTC.

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik Apr 28, 2013

Contributor

@gavinandresen ACK general roadmap

Contributor

jgarzik commented Apr 28, 2013

@gavinandresen ACK general roadmap

@johndillon

This comment has been minimized.

Show comment
Hide comment
@johndillon

johndillon Apr 29, 2013

@gavinandresen Good to hear. When people try to get the blocksize raised for the sake of penny transactions I'll be able to tell my fellow investors that people are trying to fundamentally damage Bitcoin's decentralization for the sake of penny bets.

Moore's law doesn't apply to bandwidth, and Moore's law is on it's last legs anyway.

@gavinandresen Good to hear. When people try to get the blocksize raised for the sake of penny transactions I'll be able to tell my fellow investors that people are trying to fundamentally damage Bitcoin's decentralization for the sake of penny bets.

Moore's law doesn't apply to bandwidth, and Moore's law is on it's last legs anyway.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 29, 2013

Contributor

@johndillon Seriously, take politics off github.

Contributor

petertodd commented Apr 29, 2013

@johndillon Seriously, take politics off github.

@mikehearn

This comment has been minimized.

Show comment
Hide comment
@mikehearn

mikehearn Apr 29, 2013

Contributor

Sounds good to me.

Minor detail: the formula doesn't take into account that pay-to-pubkey outputs can be spent with smaller inputs, but I suppose it doesn't matter at this point given the dominance of pay to address.

Could we bump the protocol version too? I know it's kind of a hack but it might make it easier during the deployment period - otherwise wallets might create transactions too small to get relayed if they're newer than most of their peers. I'll file a bug in bitcoinj to implement these rules.

Contributor

mikehearn commented Apr 29, 2013

Sounds good to me.

Minor detail: the formula doesn't take into account that pay-to-pubkey outputs can be spent with smaller inputs, but I suppose it doesn't matter at this point given the dominance of pay to address.

Could we bump the protocol version too? I know it's kind of a hack but it might make it easier during the deployment period - otherwise wallets might create transactions too small to get relayed if they're newer than most of their peers. I'll file a bug in bitcoinj to implement these rules.

@simondlr

This comment has been minimized.

Show comment
Hide comment
@simondlr

simondlr Apr 29, 2013

Contributor

Question. If I understand this correctly. Acceptable mintxfee/minrelayfee will now be determined by the miners (default is 0.0001 BTC)? However, currently, there would be no way to know what's the minimum fee to be reasonably included (http://bitcoin.speedstats.org/) in the next block? I see that's on the roadmap.

But for now, it would just be best to stick with 0.0001 BTC/KB for doing transactions?

Contributor

simondlr commented Apr 29, 2013

Question. If I understand this correctly. Acceptable mintxfee/minrelayfee will now be determined by the miners (default is 0.0001 BTC)? However, currently, there would be no way to know what's the minimum fee to be reasonably included (http://bitcoin.speedstats.org/) in the next block? I see that's on the roadmap.

But for now, it would just be best to stick with 0.0001 BTC/KB for doing transactions?

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Apr 29, 2013

Member

It makes no sense to bump the protocol version, as there are no protocol changes here...

Member

luke-jr commented Apr 29, 2013

It makes no sense to bump the protocol version, as there are no protocol changes here...

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 29, 2013

Contributor

@simondlr http://bitcoin.speedstats.org/ is quite wrong. The cost to send a transaction is related to it's size, not it's value. (there are some exceptions to deter abuse of free or spam transactions) Transaction fees are a process where you bid for a limited amount of blockchain space, so we can't tell you an exact number and demand for blockchain space changes day by day and hour by hour.

@mikehearn Bumping the protocol version doesn't help much because nodes can only use that information with a code change, in which case they should just follow the new rules instead.

Contributor

petertodd commented Apr 29, 2013

@simondlr http://bitcoin.speedstats.org/ is quite wrong. The cost to send a transaction is related to it's size, not it's value. (there are some exceptions to deter abuse of free or spam transactions) Transaction fees are a process where you bid for a limited amount of blockchain space, so we can't tell you an exact number and demand for blockchain space changes day by day and hour by hour.

@mikehearn Bumping the protocol version doesn't help much because nodes can only use that information with a code change, in which case they should just follow the new rules instead.

@simondlr

This comment has been minimized.

Show comment
Hide comment
@simondlr

simondlr Apr 29, 2013

Contributor

@petertodd ah yes. Of course. Thanks! Isn't the plan with the memory pool to dynamically check what tx fee (per kb) is needed to be included in the block? Thus you will know by the hour (or per block)? Or am I getting it wrong? I tried finding the discussion on the memory pool, but couldn't find it. Can someone link it to me please?

Contributor

simondlr commented Apr 29, 2013

@petertodd ah yes. Of course. Thanks! Isn't the plan with the memory pool to dynamically check what tx fee (per kb) is needed to be included in the block? Thus you will know by the hour (or per block)? Or am I getting it wrong? I tried finding the discussion on the memory pool, but couldn't find it. Can someone link it to me please?

@mikehearn

This comment has been minimized.

Show comment
Hide comment
@mikehearn

mikehearn Apr 29, 2013

Contributor

The protocol is changing because the IsStandard rules are changing. The problem is version skew. We don't know when this new code will be released and we don't know how fast people will upgrade. So my problem is, let's say we release new Android apps tomorrow that think the fee is now 0.0001/kb. People will start complaining that their sends don't go through. If there's a new protocol version the app can just measure the majority version and when it seems most nodes have upgraded, switch to the new rules and assume that 50% nodes will allow good enough propagation to the miners.

Contributor

mikehearn commented Apr 29, 2013

The protocol is changing because the IsStandard rules are changing. The problem is version skew. We don't know when this new code will be released and we don't know how fast people will upgrade. So my problem is, let's say we release new Android apps tomorrow that think the fee is now 0.0001/kb. People will start complaining that their sends don't go through. If there's a new protocol version the app can just measure the majority version and when it seems most nodes have upgraded, switch to the new rules and assume that 50% nodes will allow good enough propagation to the miners.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 29, 2013

Contributor

I think we'd all prefer that nodes just follow the new rules regardless of what the majority are doing...

That said does bitcoinj and by extension the Android wallets have any mechanism to tell the user when the majority of bitcoin nodes they are connected to are of a higher protocol version number than expected? It'd be a perfectly good "you might want to upgrade your software" mechanism, albeit one that can have false positives.

In general though we should not bake in an assumption that propagation of any given transaction is reliable enough to just send and forget. Watching for transactions coming back to you from other nodes is a very good practice. For instance, send the transaction to one peer first, and see if your other peers broadcast it to you before you try them as well.

Contributor

petertodd commented Apr 29, 2013

I think we'd all prefer that nodes just follow the new rules regardless of what the majority are doing...

That said does bitcoinj and by extension the Android wallets have any mechanism to tell the user when the majority of bitcoin nodes they are connected to are of a higher protocol version number than expected? It'd be a perfectly good "you might want to upgrade your software" mechanism, albeit one that can have false positives.

In general though we should not bake in an assumption that propagation of any given transaction is reliable enough to just send and forget. Watching for transactions coming back to you from other nodes is a very good practice. For instance, send the transaction to one peer first, and see if your other peers broadcast it to you before you try them as well.

@schildbach

This comment has been minimized.

Show comment
Hide comment
@schildbach

schildbach Apr 29, 2013

Contributor

@petertodd Nothing like that yet in Bitcoin Wallet. It updates when a new version is uploaded to the market. So this 50% switch thing could be implemented manually by uploading the right version at the right time.

Contributor

schildbach commented Apr 29, 2013

@petertodd Nothing like that yet in Bitcoin Wallet. It updates when a new version is uploaded to the market. So this 50% switch thing could be implemented manually by uploading the right version at the right time.

@mikehearn

This comment has been minimized.

Show comment
Hide comment
@mikehearn

mikehearn Apr 29, 2013

Contributor

Yeah, we already watch for propagation which helps, but if it doesn't
propagate then the user is still out of luck. We can indeed do what we did
fore bloom filtering and just wait, but even for that we had version gating
because the peer selection is sort of random. Bumping the version doesn't
cost anything so why not do it?
On 29 Apr 2013 19:20, "Peter Todd" notifications@github.com wrote:

I think we'd all prefer that nodes just follow the new rules regardless of
what the majority are doing...

That said does bitcoinj and by extension the Android wallets have any
mechanism to tell the user when the majority of bitcoin nodes they are
connected to are of a higher protocol version number than expected? It'd be
a perfectly good "you might want to upgrade your software" mechanism,
albeit one that can have false positives.

In general though we should not bake in an assumption that propagation of
any given transaction is reliable enough to just send and forget. Watching
for transactions coming back to you from other nodes is a very good
practice. For instance, send the transaction to one peer first, and see if
your other peers broadcast it to you before you try them as well.


Reply to this email directly or view it on GitHubhttps://github.com/bitcoin/bitcoin/pull/2577#issuecomment-17180671
.

Contributor

mikehearn commented Apr 29, 2013

Yeah, we already watch for propagation which helps, but if it doesn't
propagate then the user is still out of luck. We can indeed do what we did
fore bloom filtering and just wait, but even for that we had version gating
because the peer selection is sort of random. Bumping the version doesn't
cost anything so why not do it?
On 29 Apr 2013 19:20, "Peter Todd" notifications@github.com wrote:

I think we'd all prefer that nodes just follow the new rules regardless of
what the majority are doing...

That said does bitcoinj and by extension the Android wallets have any
mechanism to tell the user when the majority of bitcoin nodes they are
connected to are of a higher protocol version number than expected? It'd be
a perfectly good "you might want to upgrade your software" mechanism,
albeit one that can have false positives.

In general though we should not bake in an assumption that propagation of
any given transaction is reliable enough to just send and forget. Watching
for transactions coming back to you from other nodes is a very good
practice. For instance, send the transaction to one peer first, and see if
your other peers broadcast it to you before you try them as well.


Reply to this email directly or view it on GitHubhttps://github.com/bitcoin/bitcoin/pull/2577#issuecomment-17180671
.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 29, 2013

Contributor

Among other things it makes it easy for implementations to find nodes that don't follow the new rules to try to get their non-compliant transactions propagated. Just follow the new rules and be done with it.

Contributor

petertodd commented Apr 29, 2013

Among other things it makes it easy for implementations to find nodes that don't follow the new rules to try to get their non-compliant transactions propagated. Just follow the new rules and be done with it.

@jspilman

This comment has been minimized.

Show comment
Hide comment
@jspilman

jspilman Apr 30, 2013

No response to MeniRosenfeld's comment about colored coins? IsDust -> NonStandard ban-hammer seems unnecessarily opinionated when you can just directly charge for the resources being consumed.

If cost is a factor of bytes and UTxO, it seems reasonable that fee should be based on bytes and the UTxO growth (OutCount - InCount). Yes, if InCount > OutCount then it could offset the the byte fee.

For example, donating spare change to miners is still a good idea if you pay less overall fees for having one less output.

No response to MeniRosenfeld's comment about colored coins? IsDust -> NonStandard ban-hammer seems unnecessarily opinionated when you can just directly charge for the resources being consumed.

If cost is a factor of bytes and UTxO, it seems reasonable that fee should be based on bytes and the UTxO growth (OutCount - InCount). Yes, if InCount > OutCount then it could offset the the byte fee.

For example, donating spare change to miners is still a good idea if you pay less overall fees for having one less output.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 30, 2013

Contributor

You know, if the colored coin people can't figure out how to make colored coin systems work with this patch, screw them. I've told them how to do it right multiple times and they don't listen, which is plenty enough evidence that they're amateurs who don't understand what they're doing. So why let their lack of imagination threaten Bitcoin?

Contributor

petertodd commented Apr 30, 2013

You know, if the colored coin people can't figure out how to make colored coin systems work with this patch, screw them. I've told them how to do it right multiple times and they don't listen, which is plenty enough evidence that they're amateurs who don't understand what they're doing. So why let their lack of imagination threaten Bitcoin?

@mikehearn

This comment has been minimized.

Show comment
Hide comment
@mikehearn

mikehearn Apr 30, 2013

Contributor

Huh? If an implementation wants to try propagating a non-standard transaction it can just blast it out to half its peers and see if it hears it back on the other half. Or it can time how long it takes for those transactions to appear in the chain, or submit directly to a pool that accepts them, or many other things.

It's not like we're going to run out of version numbers. As a client implementor, I'm saying it would help in the case where release schedules aren't aligned.

Contributor

mikehearn commented Apr 30, 2013

Huh? If an implementation wants to try propagating a non-standard transaction it can just blast it out to half its peers and see if it hears it back on the other half. Or it can time how long it takes for those transactions to appear in the chain, or submit directly to a pool that accepts them, or many other things.

It's not like we're going to run out of version numbers. As a client implementor, I'm saying it would help in the case where release schedules aren't aligned.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Apr 30, 2013

Contributor

Tell you what, lets go a little farther with your idea and add a new service bit NODE_RELAYCOST. Nodes with that service bit set will also advertise the minimum txout value they will propagate, as well as the minimum fee/KB or priority/KB required for them to relay a transaction. We can then hardcode the current defaults as the assumption for nodes that do not have that bit set.

Contributor

petertodd commented Apr 30, 2013

Tell you what, lets go a little farther with your idea and add a new service bit NODE_RELAYCOST. Nodes with that service bit set will also advertise the minimum txout value they will propagate, as well as the minimum fee/KB or priority/KB required for them to relay a transaction. We can then hardcode the current defaults as the assumption for nodes that do not have that bit set.

@BlueMeanie

This comment has been minimized.

Show comment
Hide comment
@BlueMeanie

BlueMeanie Apr 30, 2013

@petertodd , with all due respect, could you please link us to your proposal re. Colored Coins? I have been participating in the discussion on the Google Group, and I pointed out the problem re. Microtransactions about a week ago. I agree with many of your stated frustrations. Thanks.

@petertodd , with all due respect, could you please link us to your proposal re. Colored Coins? I have been participating in the discussion on the Google Group, and I pointed out the problem re. Microtransactions about a week ago. I agree with many of your stated frustrations. Thanks.

@satoshiroulette

This comment has been minimized.

Show comment
Hide comment
@satoshiroulette

satoshiroulette May 9, 2013

@gmaxwell we wont be creating them for much longer ;)

Previously we used to reforge our dust into a whole piece of bitcoin again by using raw transaction api and sending to a new address in the same wallet with no fees.

Thanks for the early warning, we shall get to work updating our game configs so we are ready for this change.

@gmaxwell we wont be creating them for much longer ;)

Previously we used to reforge our dust into a whole piece of bitcoin again by using raw transaction api and sending to a new address in the same wallet with no fees.

Thanks for the early warning, we shall get to work updating our game configs so we are ready for this change.

@paraipan

This comment has been minimized.

Show comment
Hide comment
@paraipan

paraipan May 10, 2013

Contributor

NACK, you don't solve the spam problem by including more "magic" numbers

Contributor

paraipan commented May 10, 2013

NACK, you don't solve the spam problem by including more "magic" numbers

@gmaxwell

This comment has been minimized.

Show comment
Hide comment
@gmaxwell

gmaxwell May 10, 2013

Member

@paraipan There are no more magic numbers.

Member

gmaxwell commented May 10, 2013

@paraipan There are no more magic numbers.

@paraipan

This comment has been minimized.

Show comment
Hide comment
@paraipan

paraipan May 10, 2013

Contributor

Well that is strange, I though you guys called "magic" all arbitrary values in bitcoin source code.

Contributor

paraipan commented May 10, 2013

Well that is strange, I though you guys called "magic" all arbitrary values in bitcoin source code.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd May 10, 2013

Contributor

@paraipan We already have a "magic" number for minimum allowed txout value, and that magic number was 1 satoshi. This patch changes that magic number to 5430 satoshis; no new magic numbers are being added.

edit: as sipa and gmaxwell point out, this isn't strictly correct. I should have said "This patch changes the default setting of that magic number to 5430 satoshis, and additionally lets you easily change it with a configuration setting. (although, please don't unless you know what you're doing)"

Contributor

petertodd commented May 10, 2013

@paraipan We already have a "magic" number for minimum allowed txout value, and that magic number was 1 satoshi. This patch changes that magic number to 5430 satoshis; no new magic numbers are being added.

edit: as sipa and gmaxwell point out, this isn't strictly correct. I should have said "This patch changes the default setting of that magic number to 5430 satoshis, and additionally lets you easily change it with a configuration setting. (although, please don't unless you know what you're doing)"

@cjdelisle

This comment has been minimized.

Show comment
Hide comment
@cjdelisle

cjdelisle May 10, 2013

If you read the patch, you'll see that the 5420 number is not actually hard coded ("magic"), it aims to prevent payments where spending that payment requires a fee of more than 1/3 of the payment itself. Since "typical fees" are based on the number of bytes of transactions which bid for the 1MB per block limit, the number is arguably less magical than it was before.

Yes gmaxwell, I know that minimum fees are hardcoded but when we reach a point where there are more transactions than there is block space, those hard numbers will become meaningless as transactions which pay the minimum begin to be delayed/dropped.

If you read the patch, you'll see that the 5420 number is not actually hard coded ("magic"), it aims to prevent payments where spending that payment requires a fee of more than 1/3 of the payment itself. Since "typical fees" are based on the number of bytes of transactions which bid for the 1MB per block limit, the number is arguably less magical than it was before.

Yes gmaxwell, I know that minimum fees are hardcoded but when we reach a point where there are more transactions than there is block space, those hard numbers will become meaningless as transactions which pay the minimum begin to be delayed/dropped.

@gmaxwell

This comment has been minimized.

Show comment
Hide comment
@gmaxwell

gmaxwell May 10, 2013

Member

@petertodd @cjdelisle

No, not even that. The patch replaces a hardcoded base fee value with a configurable one, and makes the hardcoded dust value just be a fraction of the base fee value. So nothing is hardcoded except a fraction off a configured value...

Member

gmaxwell commented May 10, 2013

@petertodd @cjdelisle

No, not even that. The patch replaces a hardcoded base fee value with a configurable one, and makes the hardcoded dust value just be a fraction of the base fee value. So nothing is hardcoded except a fraction off a configured value...

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa May 10, 2013

Member

I
On 10 May 2013 20:44, "Peter Todd" notifications@github.com wrote:

@paraipan We already have a "magic" number for minimum allowed txout
value, and that magic number was 1 satoshi. This patch changes that magic
number to 5240 satoshis; no new magic numbers are being added.

That's not even true. It changes the minimum output side to whatever is at
least 3 times the fee necessary according to the active policy to spend it.
For a fee rule of 0.0001/kB, this results in 5240 satoshi. This 5240 itself
is not a constant.

And even better, this patch makes the fee policy rule configurable. So no,
this patch strictly reduces the magic numbers.

Member

sipa commented May 10, 2013

I
On 10 May 2013 20:44, "Peter Todd" notifications@github.com wrote:

@paraipan We already have a "magic" number for minimum allowed txout
value, and that magic number was 1 satoshi. This patch changes that magic
number to 5240 satoshis; no new magic numbers are being added.

That's not even true. It changes the minimum output side to whatever is at
least 3 times the fee necessary according to the active policy to spend it.
For a fee rule of 0.0001/kB, this results in 5240 satoshi. This 5240 itself
is not a constant.

And even better, this patch makes the fee policy rule configurable. So no,
this patch strictly reduces the magic numbers.

@paraipan

This comment has been minimized.

Show comment
Hide comment
@paraipan

paraipan May 10, 2013

Contributor

Thank for clearing it out guys.

Contributor

paraipan commented May 10, 2013

Thank for clearing it out guys.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd May 10, 2013

Contributor

@gmaxwell @sipa Sorry, I should have taken the time to write a more detailed response than my quick approximation.

Contributor

petertodd commented May 10, 2013

@gmaxwell @sipa Sorry, I should have taken the time to write a more detailed response than my quick approximation.

@jspilman

This comment has been minimized.

Show comment
Hide comment
@jspilman

jspilman May 10, 2013

@caleb: It aims to prevent payments where spending that payment requires a fee of more than 1/3 of the payment itself.

@sipa: It changes the minimum output side to whatever is at least 3 times the fee necessary according to the active policy to spend it.

Yes, and yes. Well sort of...

IsDust = 1000 * TxOutAmount / (3 * (TxOutBytes + 148)) < MinRelayFeePerKB

Easier on my eyes this way:

IsDust = TxOutAmount < 3 * MinRelayFeePerKB / 1000 * (TxOutBytes + 148)

Given nMinRelayFeePerKB of 10,000, that makes IsDust 4680 Satoshi, plus 30 Satoshi / byte for the ScriptPubKey.

Standard ScriptPubKey is 25 bytes, resulting in minimum output of 5430. But hey, a P2SH output would save you 2 bytes and lower the minimum by 60 Satoshi to 5370! However, if I understand correctly, minimum actual fees paid still assumes transactions are at least 1KB:

Line 600: int64 nMinFee = (1 + (int64)nBytes / 1000) * nBaseFee;

So in practice, the minimum standard outputs will actually cost ~2x their value in fees in order to spend them. Is there any plan to update Line 600 as follows for 0.8.2:

nMinFee = (int64)nBytes * nBaseFee / 1000;

@caleb: It aims to prevent payments where spending that payment requires a fee of more than 1/3 of the payment itself.

@sipa: It changes the minimum output side to whatever is at least 3 times the fee necessary according to the active policy to spend it.

Yes, and yes. Well sort of...

IsDust = 1000 * TxOutAmount / (3 * (TxOutBytes + 148)) < MinRelayFeePerKB

Easier on my eyes this way:

IsDust = TxOutAmount < 3 * MinRelayFeePerKB / 1000 * (TxOutBytes + 148)

Given nMinRelayFeePerKB of 10,000, that makes IsDust 4680 Satoshi, plus 30 Satoshi / byte for the ScriptPubKey.

Standard ScriptPubKey is 25 bytes, resulting in minimum output of 5430. But hey, a P2SH output would save you 2 bytes and lower the minimum by 60 Satoshi to 5370! However, if I understand correctly, minimum actual fees paid still assumes transactions are at least 1KB:

Line 600: int64 nMinFee = (1 + (int64)nBytes / 1000) * nBaseFee;

So in practice, the minimum standard outputs will actually cost ~2x their value in fees in order to spend them. Is there any plan to update Line 600 as follows for 0.8.2:

nMinFee = (int64)nBytes * nBaseFee / 1000;

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd May 10, 2013

Contributor

@jspilman By "cost to spend" we mean the marginal bytes required to spend a transaction input * the fee/kb when added to an existing transaction, like one combining a whole bunch of dust outputs together, not the cost to simply create a transaction re-spending the dust output with a single input and single output.

Contributor

petertodd commented May 10, 2013

@jspilman By "cost to spend" we mean the marginal bytes required to spend a transaction input * the fee/kb when added to an existing transaction, like one combining a whole bunch of dust outputs together, not the cost to simply create a transaction re-spending the dust output with a single input and single output.

@ABISprotocol

This comment has been minimized.

Show comment
Hide comment
@ABISprotocol

ABISprotocol May 19, 2013

Let me lay it down for @gavinandresen , @gmaxwell , @petertodd , and all you others who thought you could hammer this out on github over the holidays, close it, and then not get any objections, you know, like this:

https://bitcointalk.org/index.php?topic=196138.0;all

Some bazillion (that's not a real number) of posts later, you get the gist: People are looking at what you've done, and they are saying: WTF?

It doesn't matter what nice formulas you've come up with.

It doesn't matter if you are going to justify it by saying that this avoids the problem of bitcoin getting DDOS'd.

It doesn't matter if you felt it needed to be done for some other purpose, like "to increase potential investors' confidence" or something, ahead of #Bitcoin2013.

None of this matters.

What does matter is that this is not truly open and that there are literally millions of users (estimate, but not a bad guess) you are closing out of the system who have, and always will have, a lower figure in their bitcoin wallet that hovers somewhere between .01 BTC and .003 BTC, because they have started with a small amount from faucet and used bitcoin primarily for transaction (not for storage) and / or because they access and use sites that pay in uBTC for site visits or microtasks.

I don't think you did any kind of epic polling of the millions of users or, for that matter, the many thousands of businesses / website owners that have sprung up who pay based on bitcoin microtasks or views. I would submit that most of us who are on bitcoin today have obtained their first portion of a bitcoin not because they mined it straightaway, but because they obtained it through a BunnyRun or a website visit that paid out in uBTC for visits, views, or microtasks performed. Speaking on the latter, whether or not you consider them a significant part of the bitcoin economy is irrelevant. You have disregarded their voices by acting in an "open" forum to act in this way without a more open consultation.

Github is not a proper forum to discuss this kind of matter. It is the place where changes are ultimately made that affect the code, yes. But when such changes are so large as to concern the manner in which an entire user base can utilize an increasingly trusted method of transaction, and the development under discussion is a surprise or even completely unknown to many who are about to be affected by it, in no small part because of the behavior of the developers, this is entirely unacceptable. The behavior I am seeing here in the development of this pull request, is reminiscent of the behavior not of the open community of coders and innovators, but rather reminds me of the ITU (International Telecommunication Union), the closed-door body for which only governments have a vote.

No, this is not an issue of numbers, or formulae, or how to best prevent bitcoin from being DDOS'd.

The issue here, at its core, is was this a legitimate way to address the microtransaction issue?

Was Github truly used to solicit views and ideas of Bitcoin users?

The answer is no. A small group of developers huddled together and pushed this through. This change should be pulled back until the larger community's views can be taken into account and votes on the matter can be taken outside of Github. There are a number of online deliberative tools that are available to perform such tasks. Solicitation of ideas from a larger community, prioritization of appropriate solutions, and larger community selection through ranking or vote on which solutions are the best to address a problem facing the community, this must always come before decisions are made.

This is not like other things. You are not developing a fix for Cryptocat or talking solutions for libpurple problems in Pidgin / OTR.

There is a larger community, whose voice must be heard, outside of Github.

This is Bitcoin.

Let me lay it down for @gavinandresen , @gmaxwell , @petertodd , and all you others who thought you could hammer this out on github over the holidays, close it, and then not get any objections, you know, like this:

https://bitcointalk.org/index.php?topic=196138.0;all

Some bazillion (that's not a real number) of posts later, you get the gist: People are looking at what you've done, and they are saying: WTF?

It doesn't matter what nice formulas you've come up with.

It doesn't matter if you are going to justify it by saying that this avoids the problem of bitcoin getting DDOS'd.

It doesn't matter if you felt it needed to be done for some other purpose, like "to increase potential investors' confidence" or something, ahead of #Bitcoin2013.

None of this matters.

What does matter is that this is not truly open and that there are literally millions of users (estimate, but not a bad guess) you are closing out of the system who have, and always will have, a lower figure in their bitcoin wallet that hovers somewhere between .01 BTC and .003 BTC, because they have started with a small amount from faucet and used bitcoin primarily for transaction (not for storage) and / or because they access and use sites that pay in uBTC for site visits or microtasks.

I don't think you did any kind of epic polling of the millions of users or, for that matter, the many thousands of businesses / website owners that have sprung up who pay based on bitcoin microtasks or views. I would submit that most of us who are on bitcoin today have obtained their first portion of a bitcoin not because they mined it straightaway, but because they obtained it through a BunnyRun or a website visit that paid out in uBTC for visits, views, or microtasks performed. Speaking on the latter, whether or not you consider them a significant part of the bitcoin economy is irrelevant. You have disregarded their voices by acting in an "open" forum to act in this way without a more open consultation.

Github is not a proper forum to discuss this kind of matter. It is the place where changes are ultimately made that affect the code, yes. But when such changes are so large as to concern the manner in which an entire user base can utilize an increasingly trusted method of transaction, and the development under discussion is a surprise or even completely unknown to many who are about to be affected by it, in no small part because of the behavior of the developers, this is entirely unacceptable. The behavior I am seeing here in the development of this pull request, is reminiscent of the behavior not of the open community of coders and innovators, but rather reminds me of the ITU (International Telecommunication Union), the closed-door body for which only governments have a vote.

No, this is not an issue of numbers, or formulae, or how to best prevent bitcoin from being DDOS'd.

The issue here, at its core, is was this a legitimate way to address the microtransaction issue?

Was Github truly used to solicit views and ideas of Bitcoin users?

The answer is no. A small group of developers huddled together and pushed this through. This change should be pulled back until the larger community's views can be taken into account and votes on the matter can be taken outside of Github. There are a number of online deliberative tools that are available to perform such tasks. Solicitation of ideas from a larger community, prioritization of appropriate solutions, and larger community selection through ranking or vote on which solutions are the best to address a problem facing the community, this must always come before decisions are made.

This is not like other things. You are not developing a fix for Cryptocat or talking solutions for libpurple problems in Pidgin / OTR.

There is a larger community, whose voice must be heard, outside of Github.

This is Bitcoin.

@ABISprotocol

This comment has been minimized.

Show comment
Hide comment
@ABISprotocol

ABISprotocol May 19, 2013

@gavinandresen ~ this is, like in the other discussion at bit.ly/13WSjLC ~ exactly same sort of thing. This kind of thing (how to limit dust spam, base fee, etc, what we even would define as dust spam -- all that) needs to be discussed and deliberated by the user base outside of Github with using appropriate deliberative tools that allow the (tens, hundreds of thousands of interested users, from amongst millions) to participate with an equal voice and decide: what are their priorities, how will they define things, what will be the appropriate path forward, etc. This must be done before code changes, before commits. This is a community, this is Bitcoin, the larger community must shape what it becomes.

@gavinandresen ~ this is, like in the other discussion at bit.ly/13WSjLC ~ exactly same sort of thing. This kind of thing (how to limit dust spam, base fee, etc, what we even would define as dust spam -- all that) needs to be discussed and deliberated by the user base outside of Github with using appropriate deliberative tools that allow the (tens, hundreds of thousands of interested users, from amongst millions) to participate with an equal voice and decide: what are their priorities, how will they define things, what will be the appropriate path forward, etc. This must be done before code changes, before commits. This is a community, this is Bitcoin, the larger community must shape what it becomes.

@paraipan

This comment has been minimized.

Show comment
Hide comment
@paraipan

paraipan May 19, 2013

Contributor

Me and my social network will not upgrade the client, because I oppose this arbitrary change. You're losing it Gavin!

Let users control the limits from the GUI and the network will find it's equilibrium, you're not the one who has to make these decisions.

Contributor

paraipan commented May 19, 2013

Me and my social network will not upgrade the client, because I oppose this arbitrary change. You're losing it Gavin!

Let users control the limits from the GUI and the network will find it's equilibrium, you're not the one who has to make these decisions.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa May 19, 2013

Member

I am somewhat surprised by the outcry against this, and I wonder whether it is because of the way this decision was made, or because you believe Bitcoin should support output coins that cost ~ as much to spend as they are worth.

In any case, please realize that this is just a policy change. There are no network rules affected, and everyone is free to set whatever relay fee price they believe is right for the network. I encourage you to do (and if that means not upgrading, so be it) - that's exactly something this patch makes configurable. However, it is just a stopgap solution and the intention for the near future is replacing it with a mechanism to detect what fees get confirmed and which don't and set the relay policy based on that, so we can evolve towards a free market where users bid for space in blocks.

Member

sipa commented May 19, 2013

I am somewhat surprised by the outcry against this, and I wonder whether it is because of the way this decision was made, or because you believe Bitcoin should support output coins that cost ~ as much to spend as they are worth.

In any case, please realize that this is just a policy change. There are no network rules affected, and everyone is free to set whatever relay fee price they believe is right for the network. I encourage you to do (and if that means not upgrading, so be it) - that's exactly something this patch makes configurable. However, it is just a stopgap solution and the intention for the near future is replacing it with a mechanism to detect what fees get confirmed and which don't and set the relay policy based on that, so we can evolve towards a free market where users bid for space in blocks.

@paraipan

This comment has been minimized.

Show comment
Hide comment
@paraipan

paraipan May 19, 2013

Contributor

@sipa, in Bitcoin policy change=network rules change if I'm not mistaken. I would support said "mechanism to detect what fees get confirmed" though.

Contributor

paraipan commented May 19, 2013

@sipa, in Bitcoin policy change=network rules change if I'm not mistaken. I would support said "mechanism to detect what fees get confirmed" though.

@gmaxwell

This comment has been minimized.

Show comment
Hide comment
@gmaxwell

gmaxwell May 19, 2013

Member

@paraipan You are mistaken. I thought you understood it after this discussion with you: https://bitcointalk.org/index.php?topic=197414.msg2054475#msg2054475

Member

gmaxwell commented May 19, 2013

@paraipan You are mistaken. I thought you understood it after this discussion with you: https://bitcointalk.org/index.php?topic=197414.msg2054475#msg2054475

@paraipan

This comment has been minimized.

Show comment
Hide comment
@paraipan

paraipan May 19, 2013

Contributor

@gmaxwell I understood your position then and stopped pushing the patch based on your analysis. I didn't agree with your point though as I see personal preference as more important than maintaining a predictable fee level in the protocol.

Contributor

paraipan commented May 19, 2013

@gmaxwell I understood your position then and stopped pushing the patch based on your analysis. I didn't agree with your point though as I see personal preference as more important than maintaining a predictable fee level in the protocol.

@ABISprotocol

This comment has been minimized.

Show comment
Hide comment
@ABISprotocol

ABISprotocol May 19, 2013

@sipa you just don't get it. Re-read my original comment above. Read it again if necessary for effect.

This is reminiscent of the efforts I had (with many others) to get the International Telecommunication Union to come out into the open. (They didn't.) It was a effort of over a year resulting eventually in #OpWCIT and later which spawned #OpWTF - a smaller and more focused effort of awareness about the problems associated with closed-minded organizations and groups.

I have the feeling that just such an effort is needed to shove this development group out in the open and ensure that not only this issue is dealt with but all future issues involving bitcoin developers are not pushed in this nontransparent and backchannel way that does not take into account the voices of the internet.

This is Bitcoin.

@sipa you just don't get it. Re-read my original comment above. Read it again if necessary for effect.

This is reminiscent of the efforts I had (with many others) to get the International Telecommunication Union to come out into the open. (They didn't.) It was a effort of over a year resulting eventually in #OpWCIT and later which spawned #OpWTF - a smaller and more focused effort of awareness about the problems associated with closed-minded organizations and groups.

I have the feeling that just such an effort is needed to shove this development group out in the open and ensure that not only this issue is dealt with but all future issues involving bitcoin developers are not pushed in this nontransparent and backchannel way that does not take into account the voices of the internet.

This is Bitcoin.

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik May 19, 2013

Contributor

@idiotsabound The public website containing public source code changes cannot be considered a back channel. It is already very much out in the open. All discussions occur in public, and are publicly logged and google-able.

I agree that a continual effort is needed to educate users like yourself on how open source engineering works, and how open source engineering differs from other engineering efforts.

Contributor

jgarzik commented May 19, 2013

@idiotsabound The public website containing public source code changes cannot be considered a back channel. It is already very much out in the open. All discussions occur in public, and are publicly logged and google-able.

I agree that a continual effort is needed to educate users like yourself on how open source engineering works, and how open source engineering differs from other engineering efforts.

@n074v41l4bl34u

This comment has been minimized.

Show comment
Hide comment
@n074v41l4bl34u

n074v41l4bl34u May 19, 2013

@jgarzik First, thank you for your hard work on Bitcoin protocol.
Secondly, you write "All discussions occur in public" then I ask you where was this change discussed?
To me and most other people supporting Bitcoin, the problem with this change is not only about the code itself but mostly about unilateralism with which the decision was made.
I am, as a Bitcoin user and as a Bitcoin trader, wholeheartedly against such quick-fix changes. It seems Bitcoin is becoming just a replacement for Western Union type of transfers instead of being a full-fledged currency of the Internet with some unique features - where are contracts?
The decision making process regarding Bitcoin development is becoming a problem. It is too centralized and not democratic/market driven enough. Fees should be determined by market so developers should focus on features relating to fee management rather than hardcoding another magic number.
As Bitcoin's value is ultimately set by the market, I am personally taking some money out of Bitcoin until some long-term market solution for fees management is proposed.

@jgarzik First, thank you for your hard work on Bitcoin protocol.
Secondly, you write "All discussions occur in public" then I ask you where was this change discussed?
To me and most other people supporting Bitcoin, the problem with this change is not only about the code itself but mostly about unilateralism with which the decision was made.
I am, as a Bitcoin user and as a Bitcoin trader, wholeheartedly against such quick-fix changes. It seems Bitcoin is becoming just a replacement for Western Union type of transfers instead of being a full-fledged currency of the Internet with some unique features - where are contracts?
The decision making process regarding Bitcoin development is becoming a problem. It is too centralized and not democratic/market driven enough. Fees should be determined by market so developers should focus on features relating to fee management rather than hardcoding another magic number.
As Bitcoin's value is ultimately set by the market, I am personally taking some money out of Bitcoin until some long-term market solution for fees management is proposed.

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik May 19, 2013

Contributor

This change moves towards making fees more market driven. They were hardcoded constants before. Now users may much more easily select their desired fee level for relaying and mining with a simple configuration file change.

Contributor

jgarzik commented May 19, 2013

This change moves towards making fees more market driven. They were hardcoded constants before. Now users may much more easily select their desired fee level for relaying and mining with a simple configuration file change.

@cjdelisle

This comment has been minimized.

Show comment
Hide comment
@cjdelisle

cjdelisle May 20, 2013

What a mess!
Let me first state that my opinion is my own, I am not a bitcoin developer and I have essentially no influence within the community, my opinion is worth just as much as yours; nothing.
Furthermore, I have no interest whatsoever in getting involved in the dev community until people start getting serious about user-terrorists holding the development process hostage by throwing temper tantrums.
"This is Bitcoin." Wrong, this is one bitcoin client maintained by The Bitcoin Foundation, unless you are a member of that foundation you are not a stakeholder and your opinion is worth just as much as mine; nothing. BIP-16 was a change of network rule which required consent of the miners, this is just a change in client behavior. If you want your bitcoin client to behave differently you can use a different client, create a fork of The Bitcoin Foundation's client or bother someone else.

Developers, your patience and your desire for community process is like that of a saint, you are all wonderful people. Right now though, your good will is becoming part of the problem. Any time there is public disagreement between developers over any technical detail, someone is going to use that issue as a wedge to divide the dev community and tie up the process. In regards to developer back-channels, this is something we need more of, not less. It doesn't have to be a dark room with cigar smoke and plush curtains, a real-names-only mailing list should be more than adequate to maintain civility. Every time you so patiently try to explain the reality of the issue to the same person one more time around, you miss the point. This is not a misunderstanding, it is a negotiation and you are letting these people run roughshod over you!

What does the development process need? We needs a charismatic leader. We need someone who can source ideas and opinions from users, developers, and stakeholders alike, merge them into policy and then defend that policy with religious zealotry. We need someone with the programming skill to command respect from all developers for it is the master who leads from the front. We need someone with the credibility to arbitrate disagreements between developers without allowing them to bubble up into public view where idlers will no doubt incite them further. We need a Linus.

Thanks,
Caleb

What a mess!
Let me first state that my opinion is my own, I am not a bitcoin developer and I have essentially no influence within the community, my opinion is worth just as much as yours; nothing.
Furthermore, I have no interest whatsoever in getting involved in the dev community until people start getting serious about user-terrorists holding the development process hostage by throwing temper tantrums.
"This is Bitcoin." Wrong, this is one bitcoin client maintained by The Bitcoin Foundation, unless you are a member of that foundation you are not a stakeholder and your opinion is worth just as much as mine; nothing. BIP-16 was a change of network rule which required consent of the miners, this is just a change in client behavior. If you want your bitcoin client to behave differently you can use a different client, create a fork of The Bitcoin Foundation's client or bother someone else.

Developers, your patience and your desire for community process is like that of a saint, you are all wonderful people. Right now though, your good will is becoming part of the problem. Any time there is public disagreement between developers over any technical detail, someone is going to use that issue as a wedge to divide the dev community and tie up the process. In regards to developer back-channels, this is something we need more of, not less. It doesn't have to be a dark room with cigar smoke and plush curtains, a real-names-only mailing list should be more than adequate to maintain civility. Every time you so patiently try to explain the reality of the issue to the same person one more time around, you miss the point. This is not a misunderstanding, it is a negotiation and you are letting these people run roughshod over you!

What does the development process need? We needs a charismatic leader. We need someone who can source ideas and opinions from users, developers, and stakeholders alike, merge them into policy and then defend that policy with religious zealotry. We need someone with the programming skill to command respect from all developers for it is the master who leads from the front. We need someone with the credibility to arbitrate disagreements between developers without allowing them to bubble up into public view where idlers will no doubt incite them further. We need a Linus.

Thanks,
Caleb

@BlueMeanie

This comment has been minimized.

Show comment
Hide comment
@BlueMeanie

BlueMeanie May 20, 2013

I don't see the developers arguing over anything. The people with a bone to pick appear to be largely uninformed, uninvested end users who are upset that their particular usage scenario might be at risk. As many have already suggested, if you don't like the changes, FORK and do what you will. If you get enough people to adopt your interpretation of the rules, then it will become block chain policy. I think what the core devs understand is that without these rules, there will be no Bitcoin.

btw- a paper I recently wrote in response to some of the color coin issues.

I don't see the developers arguing over anything. The people with a bone to pick appear to be largely uninformed, uninvested end users who are upset that their particular usage scenario might be at risk. As many have already suggested, if you don't like the changes, FORK and do what you will. If you get enough people to adopt your interpretation of the rules, then it will become block chain policy. I think what the core devs understand is that without these rules, there will be no Bitcoin.

btw- a paper I recently wrote in response to some of the color coin issues.

@gavinandresen

This comment has been minimized.

Show comment
Hide comment
@gavinandresen

gavinandresen Oct 9, 2013

Contributor

@idiotsabound: this pull request is an inappropriate place to discuss the NSA/etc. I'm deleting your comment, please stay on-topic, there are plenty of places to speculate about bitcoin in general (e.g. bitcointalk forums, reddit /r/bitcoin, bitcoin.stackexchange.com, #bitcoin channel in IRC).

Contributor

gavinandresen commented Oct 9, 2013

@idiotsabound: this pull request is an inappropriate place to discuss the NSA/etc. I'm deleting your comment, please stay on-topic, there are plenty of places to speculate about bitcoin in general (e.g. bitcointalk forums, reddit /r/bitcoin, bitcoin.stackexchange.com, #bitcoin channel in IRC).

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik Oct 13, 2013

Contributor

Deleted off-topic comment, and blocked user (though that may just be a jgarzik-block and not a project-wide block).

Contributor

jgarzik commented Oct 13, 2013

Deleted off-topic comment, and blocked user (though that may just be a jgarzik-block and not a project-wide block).

@dustydoge dustydoge referenced this pull request in dogecoin/dogecoin Dec 20, 2013

Closed

Bug - Dust transactions need to be addressed #38

laudney pushed a commit to reddcoin-project/reddcoin that referenced this pull request Mar 19, 2014

Merge pull request #2577 from gavinandresen/fee_bandaid
Treat dust outputs as non-standard, un-hardcode TX_FEE constants

@MarkusTeufelberger MarkusTeufelberger referenced this pull request in bitcoin-wallet/bitcoin-wallet Aug 15, 2014

Closed

Feature request: Specify miner fee when sending #18

@ynvtlmr

This comment has been minimized.

Show comment
Hide comment
@ynvtlmr

ynvtlmr Sep 26, 2014

Potentially stupid question to the thread regarding this patch:

Was fooling around with no-fee dust-transactions, and had created one that was too insignificant by the standards discussed in the thread.
As a result, a bunch of coins are now stuck unconfirmed in limbo.

Question is:
Assuming someone pushes a no-fee dust-transaction, does it ever get verified?
If so, how long would it take?
What is the proper solution to this edge case?

Transaction:
https://blockchain.info/tx/2bba1468f81540a83d31ef55ae1e080947f78b22c173d61907b563fa76ef6fc5

ynvtlmr commented Sep 26, 2014

Potentially stupid question to the thread regarding this patch:

Was fooling around with no-fee dust-transactions, and had created one that was too insignificant by the standards discussed in the thread.
As a result, a bunch of coins are now stuck unconfirmed in limbo.

Question is:
Assuming someone pushes a no-fee dust-transaction, does it ever get verified?
If so, how long would it take?
What is the proper solution to this edge case?

Transaction:
https://blockchain.info/tx/2bba1468f81540a83d31ef55ae1e080947f78b22c173d61907b563fa76ef6fc5

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Sep 26, 2014

Member

@ynvtimr You can just forget sending it, and regard the coins as not stuck. Most nodes will not even have let that transaction into the mempool, so will not regard another use of them as a double spend.

Member

laanwj commented Sep 26, 2014

@ynvtimr You can just forget sending it, and regard the coins as not stuck. Most nodes will not even have let that transaction into the mempool, so will not regard another use of them as a double spend.

@ynvtlmr

This comment has been minimized.

Show comment
Hide comment
@ynvtlmr

ynvtlmr Sep 29, 2014

Makes sense.
I figured that was likely the case as the transaction maintained 0% network propagation 4 days after the transaction date.

ynvtlmr commented Sep 29, 2014

Makes sense.
I figured that was likely the case as the transaction maintained 0% network propagation 4 days after the transaction date.

@ABISprotocol

This comment has been minimized.

Show comment
Hide comment
@ABISprotocol

ABISprotocol Sep 30, 2014

'Uneconomic dust' was indicated in this pull as "5430 satoshis, about $0.007.." at the time this was opened in April 2013 (note however that per @sipa ~ the meaning of 'uneconomic dust' is actually "3 times the cost (according to your own node's relay rules, which are configurable) of the number of bytes that are added to the blockchain by the relevant output plus how many are needed to spend it," and that this is anticipated to change in future versions). Since then a lot has changed, floating fees were announced in July of 2014 ~ though they are not yet realized. @ynvtlmr you mentioned originally, no-fee dust transactions, which reminded me of a collaboration between Coinbase and Bitmonet which was in January of 2014. That was off-chain no-fee, so very small transactions could technically be handled. This also allowed for micro-transactions on mobile (initially, Android) and other things (off blockchain micro-transactions between Coinbase users would appear to be free to the user). The question seems to be how would this be done on the network and the answer is, I think, tiny transactions cannot exist alone.

Many 'tiny transactions' (we could call it a micro-donation, too) could be bundled in a wallet ~ some concepts and ideas for that are shown here. Apart from that, more specific to the question floated above,

"Assuming someone pushes a no-fee dust-transaction, does it ever get verified?"

The answer would seem to be no, if it is truly no-fee and is in the network, but I say "seem to be," because I imagine the idea of a "wallet dusting feature" ~ as expressed by @gmaxwell ~ or a "specialized collector" to process small change and tiny txouts, which potentially could be used (in part) to gather dust together to provide for the fee in the floating or smart fee framework within the context of 0.10. (Thus transactions could be given priority which is variable depending upon conditions.)

Examining many small transactions together with a fee, and assuming a sort of bundling of many tiny transactions, one can imagine all kinds of possibilities in Core especially with the floating or smart fee framework.

'Uneconomic dust' was indicated in this pull as "5430 satoshis, about $0.007.." at the time this was opened in April 2013 (note however that per @sipa ~ the meaning of 'uneconomic dust' is actually "3 times the cost (according to your own node's relay rules, which are configurable) of the number of bytes that are added to the blockchain by the relevant output plus how many are needed to spend it," and that this is anticipated to change in future versions). Since then a lot has changed, floating fees were announced in July of 2014 ~ though they are not yet realized. @ynvtlmr you mentioned originally, no-fee dust transactions, which reminded me of a collaboration between Coinbase and Bitmonet which was in January of 2014. That was off-chain no-fee, so very small transactions could technically be handled. This also allowed for micro-transactions on mobile (initially, Android) and other things (off blockchain micro-transactions between Coinbase users would appear to be free to the user). The question seems to be how would this be done on the network and the answer is, I think, tiny transactions cannot exist alone.

Many 'tiny transactions' (we could call it a micro-donation, too) could be bundled in a wallet ~ some concepts and ideas for that are shown here. Apart from that, more specific to the question floated above,

"Assuming someone pushes a no-fee dust-transaction, does it ever get verified?"

The answer would seem to be no, if it is truly no-fee and is in the network, but I say "seem to be," because I imagine the idea of a "wallet dusting feature" ~ as expressed by @gmaxwell ~ or a "specialized collector" to process small change and tiny txouts, which potentially could be used (in part) to gather dust together to provide for the fee in the floating or smart fee framework within the context of 0.10. (Thus transactions could be given priority which is variable depending upon conditions.)

Examining many small transactions together with a fee, and assuming a sort of bundling of many tiny transactions, one can imagine all kinds of possibilities in Core especially with the floating or smart fee framework.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Sep 30, 2014

Member

Uneconomic dust is not defined as a specific number of satoshis.

It is 3 times the cost (according to your own node's relay rules, which are
configurable) of the number of bytes that are added to the blockchain by
the relevant output plus how many are needed to spend it.

In future versions the relay rules may be estimated from transactions in
the network, rather than being an (overridable) default.

Member

sipa commented Sep 30, 2014

Uneconomic dust is not defined as a specific number of satoshis.

It is 3 times the cost (according to your own node's relay rules, which are
configurable) of the number of bytes that are added to the blockchain by
the relevant output plus how many are needed to spend it.

In future versions the relay rules may be estimated from transactions in
the network, rather than being an (overridable) default.

@ABISprotocol

This comment has been minimized.

Show comment
Hide comment
@ABISprotocol

ABISprotocol Oct 2, 2014

@sipa Making correction to my prior comment, thanks for your clarification.

@sipa Making correction to my prior comment, thanks for your clarification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment