Change the default maximum OP_RETURN size to 80 bytes

[Flavien]

The maximum size for OP_RETURN outputs used to be 80 bytes, then got changed to 40 bytes to be on the safe side. We have now been running with 40 bytes for about 9 months, and nothing catastrophic happened to the Blockchain, so I am proposing to increase it back to 80 bytes.

Also, that value is now configurable through the datacarriersize option, so miners who want to stay on 40 bytes (or any other value) can easily do so.

[petertodd]

There's unit tests that you have to modify as well; try running src/test/test_bitcoin and you'll see them fail.

[Flavien]

Thanks Peter, I will fix the unit test.

[luke-jr]

Don't-care-ACK once tests are fixed.

[luke-jr]

Minor idea: Perhaps we should set the default to a random value between 0-80 at startup, to incentivise miners making their own decision without actually forcing it explicitly.

[TheBlueMatt]

This is semantically similar to #5075, and, as discussed there, this is not something to be discussed in a pull request, please propose this on the ML first, and then come back here.

[dexX7]

Let me play devil's advocat and ask: why should one use OP_RETURN at all? As long as alternatives are cheaper (and faster at this point), there is not much inceive to do so.

I took a look at cost as transaction size in relation to payload sizes and it turns out 40 byte OP_RETURN isn't really a great choice.

80 byte however, assuming one pay-to-pubkey change output, beats efficient bare multisig encoding (that is: spending multisig outputs!).

[luke-jr]

Well, that sounds like an argument for #5231 since bare multisig is not for data storage.

[dexX7]

I agree that bare multisig encoding should be discouraged as data carrier, if this PR is accepted.

Though it should be evaluated, if there is still need for #5231 then, or if it would just hurt the remaining other "legitimate" use cases.

[jgarzik]

OP_RETURN is pruneable

[laanwj]

As said on the mailing list, I'm OK with this. It makes OP_RETURN more attractive to use in favor of messier, unprunable approaches.

[gavinandresen]

utACK

[jgarzik]

ut ACK -- will close my own PR in preference to this one, after this merges, as this seems to have greater consensus.

[sipa]

I'm in the middle here. If this is because the alternative is people storing data in other non-prunable ways anyway, then it's obviously a win. For some use cases, there is no way around that.

But for many there is, and this is just providing them with an easier path that may result in an ecosystem with higher costs for everyone. I don't like encouraging that, and I don't think it's in the best interest of Bitcoin nodes to help people do so.

[gmaxwell]

I think OP_RETURN has shown itself to be seriously problematic; and we continue to have problems with people beleving that storing non-bitcoin related data in the chain (as opposed, e.g. to simple commitments or things like ECDH nonces) is an approved, correct, non-antisocial use of the system. We have people selling insane data storage services, etc. It's a bad place to be.

Meanwhile, many of the externalized cost creating services which could use op_return (e.g. data they're encoding is small) still don't bother. Rather than a success, I think it's hard to say that it hasn't been much of a success. Though: at least a few things that were going to encode data did switch to hashes, so I think the limitations were at least somewhat informative.

That said, there are some cases where I think more data can make sense and be useful. For example, I think of the ECDH address negoiation stuff isn't a terrible use... a bit inefficient compared to using an external system, but not unreasonable... and that application wants the ability to push in additional payment identifiers and the like (see petertodd's latest update to the stealth address additions).

So, I'd prefer to make any increase take the form of allowing more pushes, e.g. you can have up to {limit} bytes, in the form of pushes up to 40 bytes. Beyond furthering the message that this isn't an endorcement for externalizing your storage and transmission costs to the increasingly shrinking public network that our decenteralization depends on; this option also is permissive enough that there isn't a {well you can use checkmultsig instead} that is more permissive.

If people who would support this would also support that, I'll submit a PR.

[jgarzik]

@gmaxwell That's what #5075 does

[luke-jr]

@jgarzik No, #5075 allows multiple outputs. What @gmaxwell is suggesting is still a single output, but with multiple pushdata opcodes.

@gmaxwell Don't-care-ACK I gave this applies to your version as well.

[petertodd]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

@jgarzik No, #5075 allows multiple outputs. What @gmaxwell is
suggesting is still a single output, but with multiple pushdata
opcodes.

I have a pull-req open to allow that.

As for this pull-req, utACK
-----BEGIN PGP SIGNATURE-----
Version: APG v1.1.1

iQFQBAEBCAA6BQJUgM3EMxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8
cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhexHB/4xpXq3P3+AC3wUsJID
YgI/oLJIOS74mVYobmfi3qjF8TWtsi6IHpHKAPRS1JCakSSFRa8wviLfwvrrglYv
wW3ZusJXxjHnMsJblbhMMJl+2mF1W2RIMoYi2jiIhz4nMMFsrU6rl8vvVERXKRec
WslksFr23Tx441FCVk4K9LpmzjkeCqr5NxUGKaudVKTM9RqSId+frsGF+Teo2uNx
QVtprvTzvxqHShSEKGL+y+cMlaWZ2p8HH3nXtN1tkoEqxdqb0LEauV8Hk0MSkRz2
6kXZnFTtxa+alwVzfsEdR6mU/5cl9Sg/RmpbQraUYUAw6SEtytVeUSAwY2NfO87V
xJsb
=ioIv
-----END PGP SIGNATURE-----

[gavinandresen]

untested ACK.

So we've got three different suggestions for how to do this, and two "maybe it is a bad idea to do it at all."

Of the three suggestions: I like this one best, it is simplest conceptually.

And I still think the benefits of a prunable 80-byte OP_RETURN (and 40-byte OP_RETURN) outweigh the costs.

[petertodd]

Removing the single PUSHDATA restriction is orthogonal to how many bytes are allowed, and mostly orthogonal to how many OP_RETURN outputs are allowed.

On 4 December 2014 21:35:19 GMT+00:00, Gavin Andresen notifications@github.com wrote:

untested ACK.

So we've got three different suggestions for how to do this, and two
"maybe it is a bad idea to do it at all."

Of the three suggestions: I like this one best, it is simplest
conceptually.

And I still think the benefits of a prunable 80-byte OP_RETURN (and
40-byte OP_RETURN) outweigh the costs.

---

Reply to this email directly or view it on GitHub:
#5286 (comment)

[petertodd]

@gmaxwell

this option also is permissive enough that there isn't a that is more permissive option.

Creating unspendable outputs is another option that is more permissive. I'm of course supporting that in the coloured coin standard I'm working on, an approach being taken by many projects. (that particular application has no show-stopper issues even with P2SH²)

[laanwj]

I think OP_RETURN has shown itself to be seriously problematic; and we continue to have problems with people beleving that storing non-bitcoin related data in the chain (as opposed, e.g. to simple commitments or things like ECDH nonces) is an approved, correct, non-antisocial use of the system

I fully agree. However, practically, until the time that we have a solution for spam in unspendable outputs (cluttering the UTXO set which is so much worse), at least encouraging the least antisocial method for people that insist on doing it is advantageous, I'd suppose...

[dexX7]

cluttering the UTXO set which is so much worse

I believe an increase in OP_RETURN's default size would mainly serve as inceive to move away from bare multisig encoding which is currently "cheaper". Getting the attention of (ab-) users who create unspendable outputs, such as cryptograffiti.info, appears to be another topic and an increase to 80 byte probably has no effect in that context.

[gmaxwell]

Fake h160 and checkmultisig stuffing all also are limited to storing <32 bytes per push, which is why I pointed out that the limited OP_RETURN pushes would still be "prefered", but still lilmiting the people promoting misuse of the system from claiming that arbritary data storage is a endorsed, supported, and maintained activity.

[petertodd]

@gmaxwell The relevant metric is cost per byte published; I'd suggest you get the recent P2SH IsStandard() rule removal reverted if you are concerned about sending such a "message" and wish to be self-consistent; see https://github.com/petertodd/uniquebits specifically https://github.com/petertodd/uniquebits/blob/master/uniquebits/_unibits.py#L85

@dexX7 How was that graph calculated?

[dexX7]

Hey @petertodd: the chart was created based on the following assumptions:

• no unspendable outputs
• after each message, the state should be similar (means in pracise there might be change)
• all public keys are compressed
• if public keys are used to embed, then two bytes are subtracted to form fully valid public keys
• if the payload size surpasses capacity, then multiple transactions are created
• pay-to-pubkey is cheaper than pay-to-pubkeyhash

Please let me know, if there are flaws or more efficient ways to transport data, but I came up with the following scripts:

OP_RETURN: up to 40 byte (80 byte, 200 byte)

scriptSig: [signature]
scriptPubKey 0: [op_return] [payload]
scriptPubKey 1: [public_key] [op_checksig]

Bare multisig: up to 62 byte, embedded as public key, 31 byte each, with n = [2, 3]

scriptSig: [op_0] [signature]
scriptPubKey: [op_1] [public_key] [payload] ( [payload] ) [op_n] [op_checkmultisig]

P2SH: up to 434 byte, embedded as public key, 31 byte each

Transaction 1 (commit):
scriptSig: [signature]
scriptPubKey: [op_hash160] [script_hash] [op_equal]

Transaction 2 (reveal) with n = [2, 15]:
scriptSig: [op_0] [signature] [op_1] [public_key] [payload] ( ... ) [op_n] [op_checkmultisig]
scriptPubKey: [public_key] [op_checksig]

[petertodd]

no unspendable outputs

Better to determine what the cost for creating them is due to coins lost to the dust limit. Also calculate w/o dust limit, as lots of hashing power doesn't follow it.

if the payload size surpasses capacity, then multiple transactions are created

Why multiple transactions? Why not have multiple outputs instead?

[bharrisau]

Why multiple transactions? Why not have multiple outputs instead?

Only one OP_RETURN output per transaction?

[petertodd]

On Mon, Feb 02, 2015 at 08:43:08PM -0800, Gregory Maxwell wrote:

@genjix PeterTodd specced out an alternative that can just put many prefixes in a single output.

That alternative compromises on privacy for no good reason; I specced it
only because you asked me too and said you'd do a pull-req with the
necessary op-ret changes to make it work. You haven't done that, so I'm
assuming you're not interested. In the meantime, I'd much rather use the
version that doesn't have silly compromises.

[btcdrak]

Can this be backported to 0.10?

[petertodd]

@btcdrak You can submit a pull-req directly to the v0.10 branch actually. I could do that in another day or two; time-sensitive so probably better if you do.

[gidgreen]

Some empirical data from the coinsecrets.org database to contribute to the discussion. So far there have been 16154 OP_RETURNs on mainnet, as of block 341912.

Breakdown by protocol that we can identify:
4877 Open Assets
3015 Proof of Existence
1933 CoinSpark
600 BlockSign
5729 (other)

So for now the two main identifiable use cases are colored coins and hashing a document to timestamp its existence. There are still far fewer OP_RETURNs than multisigs used by Counterparty.

Average OP_RETURNs per block (grouped by 10,000 blocks):
20xxxx 0.0000
21xxxx 0.0000
22xxxx 0.0002
23xxxx 0.0000
24xxxx 0.0029
25xxxx 0.0127
26xxxx 0.0134
27xxxx 0.0195
28xxxx 0.0274
29xxxx 0.0332
30xxxx 0.1520
31xxxx 0.2667
32xxxx 0.4258
33xxxx 0.5170
34xxxx 0.7559

So there is clear growth but it's not showing signs of being exponential.

[laanwj]

@gidgreen Interesting numbers, thanks for the research!

[jtimon]

Proof of existence doesn't need op_return, seehttps://github.com/Blockstream/contracthashtool . In fact, there's a form of hard to censor colored coins that uses a similar technique instead of putting data in the chain (I believe the most used name is "committed assets".

If this gets into 0.10 I will be jealous. Why this can go in but not my commit removing the miner's hashmeter, for example?

[petertodd]

On 4 February 2015 07:40:13 GMT-08:00, "Jorge Timón" notifications@github.com wrote:

Proof of existence doesn't need op_return,
seehttps://github.com/Blockstream/contracthashtool .

That technique is quite dangerous to use, as it creates coins unrecoverable by your wallet seed.

If this gets into 0.10 I will be jealous. Why this can go in but not my
commit removing the miner's hashmeter, for example?

Because more people cared about this.

[btcdrak]

@jtimon I think the reasoning for adding this patch to 0.10 is much more straight forward: it limits "damage" being done by meta protocols that currently use more wasteful and bloaty encoding. Allowing them to move to OP_RETURN encoding sooner is better for everyone.

Having this go into 0.10 now would mean the eco system moves to relaying 80 opreturns faster than if we wait for 0.11 which might be 6 months away. Let's face it, wide deployment of a new version takes considerable time after release, so sooner is better for everyone. Given that the change is uncontroversial technically speaking, I personally think it should go into 0.10 now.

@petertodd I didn't create a PR for this because it's a straightforward cherry-pick, and since all merges are done manually anyhow, it doesn't create any more or less work for the maintainers to pull.

[jtimon]

I just said I will be jealous...

@petertodd when I said "proof of publication" I really meant proof of existence/timestamping (the use case @gidgreen was referring to). I don't understand the danger, you can pay to yourself to do this and you have all the information you require to claim your coins once again.

[gmaxwell]

@petertodd The contract hash tool works just as well on DSA nonces.

[gmaxwell]

@btcdrak One of the things requests is actual example applications; which have been in short supply. It's hard to not get the impression that people are not keeping their applications secret in order to game the process here.

[jgarzik]

This uses proof of publication: https://en.bitcoin.it/wiki/Identity_protocol_v1 Admittedly the proof part is not really used at all today, for limit related reasons.

[petertodd]

On 4 February 2015 10:44:55 GMT-08:00, Gregory Maxwell notifications@github.com wrote:

@petertodd The contract hash tool works just as well on DSA nonces.

Mind uploading an implementation then? Also last I checked with you that crypto was uncertain; a concern repeated by some non-Bitcoin cryptographers I ran the idea past.