BIP 322: Generic Signed Message Format

[kallewoof]

Formatted version: https://github.com/kallewoof/bips/blob/bip-generic-signmessage/bip-0322.mediawiki

[sipa]

Does the scriptPubKey need to be included? That can be derived from the address to check it against.

[odarboe]

Suggested improvement:
Line 108: Retain the message as is for all sighash operations (i.e. all sign and verify operations should sign and verify the entire list of UTXO:s ...........Taking a sub-set of the proofs and turning them into a new proof should not be valid.

or
Line 108: Retain the message as is for all sighash operations (i.e. all sign and verify operations should validate the entire list of UTXO:s ...........Taking a sub-set of the proofs and turning them into a new proof should not be valid.

[kallewoof]

Thanks, will tweak!

[jl2012]

This scheme might not be compatible with OP_CODESEPARATOR. While it is rarely used, it changes scriptCode and affects the signature validity. So you need to concatenate scriptCode with the message in order to fully support OP_CODESEPARATOR

A related problem is since your message does not commit to the public key, it is possible to "recover" the public key from any random signature and random message, with no one has the private key. (I think the existing message system has the same problem)

I also have a related idea: turn an unused opcode (e.g. 0xf0) to OP_MESSAGEONLY. The code remains invalid in bitcoin txs and requires no change in consensus. In message system, however, it is interpreted as OP_NOP. So this allows people separating their message keys and money-holding keys. With MAST, the use of OP_MESSAGEONLY could be totally invisible except to the message validators.

[instagibbs]

privkeys?

[kallewoof]

There is one privkey for each scriptPubKey I assume (this is in the individual case, even though it is tracking the SPK's in the SPK set for early abort).

[maaku]

Why not construct a bitcoin-like transaction instead of something custom? Make a fake tx with the address as the output and a fake tx spending that output with the hash of the message in an OP_RETURN. That way existing hardware signers and other infrastructure could sign messages without changes.

[jl2012]

Why not construct a bitcoin-like transaction instead of something custom? Make a fake tx with the address as the output and a fake tx spending that output with the hash of the message in an OP_RETURN.

I agree with @maaku , except that the SignatureHash() must be slightly modified to make sure that the "fake tx" couldn't be valid on the mainnet. For example, replace nSigHash with nSigHash|0xff000000

That way existing hardware signers and other infrastructure could sign messages without changes.

This is very dangerous as the hardware wallet couldn't tell if they are signing a message or a tx. Changes are needed, just as minimal as possible.

[jl2012]

using plain double-sha256 here is very dangerous, as attackers could trick people to sign real txs this way. You need to either pad something before or after the message, or simply use a different hash function.

for example, it could be sighash = sha256(sha256(msg)|0xff).

If the size of msg is small enough to be shown on the wallet screen, the wallet will ask user to confirm the message and will do the whole double hash

If the size of msg is too big, the untrusted computer will send sha256(msg) to the wallet, and the wallet will only do the second hash. Careful users will use a trusted machine to verify the hash (they don't need special software because it's simple sha256). Even not verified, the worst case would be signing a wrong message, not sending bitcoin to attacker

[kallewoof]

The msg is prefixed with the string sequence "Bitcoin Message:", so that should not be a problem.

I initially went for single sha256 but realized they could then potentially pass the single sha256 of a transaction and it would thus become the actual txid. But that relies again on the "Bitcoin Message:" part being ignored.

[kallewoof]

As for a long message, perhaps there should be a limit on message size. I don't think I understand the benefits of single-sha256'ing and showing that. It doesn't seem like it would convey anything at all to the user.

[jl2012]

The msg is prefixed with the string sequence "Bitcoin Message:", so that should not be a problem.

This is 16 bytes at the beginning. To attack it, a 12 bytes (96 bit) collision is needed. Bitcoin mining is now 82bit/day, so it is 45 years for 96 bit. Not likely, but also not impossible.

The only way to make it completely safe (in terms of legacy sighash and BIP143), is to put a non-zero byte at the end of the message.

I don't think I understand the benefits of single-sha256'ing and showing that. It doesn't seem like it would convey anything at all to the user.

Maybe, but this is what Ledger Wallet is currently doing.

[kallewoof]

@jl2012

This is 16 bytes at the beginning. To attack it, a 12 bytes (96 bit) collision is needed. Bitcoin mining is now 82bit/day, so it is 45 years for 96 bit. Not likely, but also not impossible.

The only way to make it completely safe (in terms of legacy sighash and BIP143), is to put a non-zero byte at the end of the message.

Right now, the sighash is defined as sha256(sha256(scriptPubKey || pre-image)) where pre-image is "Bitcoin Message:" for a regular signmessage call. The added scriptPubKey should address the problem, right?

Maybe, but this is what Ledger Wallet is currently doing.

Interesting. I didn't realize this. I will have to read up on why they do that.

[maaku]

Modifying the signature hash defeats the point of having drop in hardware signer support. Make the setup fake transaction a coinbase or otherwise give it a fake input that cannot conform.

[luke-jr]

The point of hardware signers is to confirm (and perhaps track) spending amounts. A signed message, however, should not present as a spend. So either way, the hardware needs to explicitly support signing messages - and therefore IMO a modified signature hash is fine.

[luke-jr]

This should just be part of the specification IMO, providing a clear distinction between proof-of-funds (which only verifies an amount, not address(es)) and proof-of-receiving-at-address.

[kallewoof]

I was aiming for a simple and lean proposal and this was only meant as a "you CAN do this, by extending this in the future", but it turned into a more complete proposal than I intended. You're probably right.

[kallewoof]

I've rewritten the proposal to include this as part of the proposal.

[luke-jr]

Should elaborate on what is being signed/verified.

[kallewoof]

The two sections further down are meant to do that. Is something missing?

[kallewoof]

@sipa

Does the scriptPubKey need to be included? That can be derived from the address to check it against.

The idea here is to only include the scriptPubKey. The conversion from address to scriptPubKey would be done by the UI/IX. This means a signmessage implementer could let users sign for a specific address ("message") or a txid:vout ("POF") or something else from which scriptPubKeys can be derived.

[kallewoof]

@jl2012

This scheme might not be compatible with OP_CODESEPARATOR. While it is rarely used, it changes scriptCode and affects the signature validity. So you need to concatenate scriptCode with the message in order to fully support OP_CODESEPARATOR

The implementation (for Bitcoin Core) would literally add a new BaseSignatureChecker class that takes a predefined sighash (the message sighash) and executes VerifyScript as normal. Would that actually be affected by OP_CODESEPARATOR in any way? The sighash is the message only, so it shouldn't matter, right?

A related problem is since your message does not commit to the public key, it is possible to "recover" the public key from any random signature and random message, with no one has the private key. (I think the existing message system has the same problem)

So, include the scriptPubKey in the sighash, then?

I also have a related idea: turn an unused opcode (e.g. 0xf0) to OP_MESSAGEONLY. The code remains invalid in bitcoin txs and requires no change in consensus. In message system, however, it is interpreted as OP_NOP. So this allows people separating their message keys and money-holding keys. With MAST, the use of OP_MESSAGEONLY could be totally invisible except to the message validators.

I love the idea, but not sure if it's worth using an entire op code when you can probably solve it with simple message prefix and/or (as above) a custom sighash format (including scriptPubKey).

[kallewoof]

@maaku

Why not construct a bitcoin-like transaction instead of something custom? Make a fake tx with the address as the output and a fake tx spending that output with the hash of the message in an OP_RETURN. That way existing hardware signers and other infrastructure could sign messages without changes.

It feels like unnecessary overhead, when all you really want to do is

• for a given scriptPubKey P
• given a message M
  • create a signature S that signs M for P
  • prove/verify that S signs M for P

Why would you even bother with transactions in this case? It adds unnecessary data to the proof, and unnecessary complexity to the prover/verifier.

[sipa]

@kallewoof But I don't understand why you're including the scriptPubKey in the signature. Transactions don't have that either. You verify them against a scriptPubKey.

[kallewoof]

@sipa Oh, I see what you mean. Yeah, I think it makes sense to require the verifier/prover to retain the order of the SPKs which means they are not needed in the proof. Fixing.

[maaku]

@kallewoof It shouldn't add any overhead as far as I can tell. It's just a different convention: this BIP adds "Bitcoin Message:" to the beginning and hashes. What I suggest above would have you deterministically create a sequence of two bitcoin transactions, the first containing an output with the scriptPubKey, and the second "spending" it. In a slight revision of my earlier suggestion, the salted hash of the message to be signed would be placed in the input.txid of the first transaction, thereby preventing it from ever being interpreted as a valid transaction.

Yes this is more data structures constructed in memory to fake something that looks like a real transaction, but it's all deterministically constructed using fixed rules, and isn't included in the proof any more than the string "Bitcoin Message:" is in yours.

[kallewoof]

I see what you're saying, and I'm not really against it, but I would love if someone could explain why it is better than just a sighash.

[maaku]

@kallewoof how do you get your hardware wallet or corporate HSM to sign your scheme, without a firmware update from the manufacturer?

[kallewoof]

@maaku I flat out assumed a firmware update would be necessary to support this. I'm definitely intrigued if it is possible to support this with NLS'es and Trezor's without an actual firmware update!

[maaku]

The point is you make it look like a bitcoin transaction, you sign it like a bitcoin transaction, you verify it like a bitcoin transaction, and nothing in the signing path even needs to be aware it is not actually a bitcoin transaction.

It also works well with proof of reserve: the proof of reserve is a bitcoin transaction spending all the funds, but with an additional input (covered by SIGHASH_ALL) that points to a fake/invalid tx. This has the additional benefit of working in a forward compatible way with any future bitcoin extension, like confidential transactions or mimblewimble: your proof of reserve could have blinded inputs and outputs as well, or whatever else the bitcoin protocol is made to allow. As long as the spends are tangled up with the fake input (via SIGHASH_ALL or a mimblewimble kernel, or whatever), it doesn't matter.

[kallewoof]

@maaku Thanks a lot for the explanation. That definitely sounds worth it!

[kallewoof]

@maaku After discussion with others on IRC, the forward compatibility part is pretty nice, but there are also dangers with a specification where the signers can be potentially fooled into signing one thing thinking it's something else.

With the approach you are suggesting, it would be impossible for existing HW wallets (without a firmware upgrade) to distinguish between messages and transactions, meaning there is no way for the HW wallet to inform the user that they're signing a message vs signing a transaction.

[maaku]

Yes I'm aware of that. I'm not suggesting it as a desirable end-goal in the sense that hardware wallets should be blind to message signing. Rather it is a particularly nice feature to have now as there are hardware wallets and HSMs already deployed which won't be receiving field upgrades, either from lack of upstream support or because they are explicitly designed NOT to in the corporate HSM setting. Signing things not understood at the time of firmware burn-in and key loading would otherwise be an attack vector for emptying funds held by the HSM. (But there are enough ways to make a transaction invalid that the HSM would see it as a signed transaction which never confirms and is double-spent according to whatever protocol governs the HSM.) There is even greater need for this with respect to corporate HSMs doing proof of reserve, then raw message signing.

Beyond being a good transition mechanism now, the same argument holds for any future bitcoin protocol upgrade. If you add confidential transactions, whatever general signing/proof-of-reserve system you come up with now would have to be updated to support CT. If you use (fake) bitcoin transactions, you get it for "free," on any device that gets the CT upgrade.

[maaku]

Oh I probably should mention this explicitly: to address the "sign something that looks like a transaction" concerns you can make the transaction pay to the same owner, so any signing interface shows a "0 btc" transaction.

This is probably needed anyway to get around any signing logic restrictions enforced by the HSM.

[jl2012]

@maaku: being unable to upgrade is a feature, not a bug. If the firmware designer wanted to allow message signing, it would have been already supported. If not, I'd assume that's intentional.

The whole point of this BIP is to define a generic format, which could cover any previous and future scripts. For an unupgradable device, however, everything are set in stone so forward compatibility makes no sense to them

[maaku]

Not on some of the commercial HSM systems I’ve actually worked on at least, where an essential part of the security model is the inability to change the signing logic of the device once it is switched on and the keys generated — signing logic governing what sorts of signatures the device is allowed to create. Otherwise you’re just one firmware update away from signing anything, including a steal-all-the-monies transaction, which introduces a central point of failure with however manages the firmware update process. And I simply disagree on forward compatibility making no sense, and this BIP points to examples which disprove that—proof of reserve messages. An exchange with an HSM rate limiting withdraws is exactly the situation where in you might want to add this message signing feature to an already deployed high-value HSM which takes the extra security precautions of dis-allowing live updates to its signing authorization logic. So you masquerade the proof-of-reserve message to look like a zero-fee transaction to itself, which it IS authorized to sign.

…

On Sep 11, 2018, at 12:48 AM, Johnson Lau ***@***.***> wrote: @maaku <https://github.com/maaku>: being unable to upgrade is a feature, not a bug. If the firmware designer wanted to allow message signing, it would have been already supported. If not, I'd assume that's intentional. The whole point of this BIP is to define a generic format, which could cover any previous and future scripts. For an unupgradable device, however, everything are set in stone so forward compatibility makes no sense to them — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#725 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEOIov1q8QujVZ7j_AmQtx7LDO3WfEaks5uZ2regaJpZM4Wg57j>.

[prusnak]

First, thanks for the draft.

Second, there is already a way how to sign/verify messages via Segwit-in-P2SH and Native-Segwit addresses: see bitcoin/bitcoin#10542 (comment)
We are using this solution in Trezor for almost a year now and other wallets use this too already.

Third, I am against implementing generic SignMessage via a specially crafted transaction. This will severely cripple UX for hardware wallets that actually show the contents of the transaction during the signing. If the usecase is proof-of-funds, that might be the option, though, but for nothing else.

[jl2012]

@maaku If the HSM designer wanted the proof-of-reserve feature, they would/should have done it already (e.g. opendime). If they did not include this feature at the design stage, I'd assume it's intentional.

Also, there is no guarantee that whatever you propose would be compatible with every existing HSM. It totally depends on the policy of the HSM. For example, it might not allow inclusion of an OP_RETURN or any non-whitelisted scriptPubKey, even if it is 0 value, so you have nowhere to hide your message.

[kallerosenbaum]

I thought the witness program was the 2-40 bytes after the witness version in the scriptPubKey. It should be the witness, or possibly witness field here, no? If so, several places from here on should change.

[kallewoof]

I'm not entirely sure what you are suggesting, but the scriptPubKey is <version> <32 byte hash> where the latter is a hash of the witness program. The witness program itself is in the spending transaction's input, not in the scriptPubKey. The scriptPubKey simply commits to it.

[kallerosenbaum]

BIP141 says the hash IS the witness program. I asked about this on twitter (https://twitter.com/kallerosenbaum/status/949630404302196736) and @sipa explained it well in that thread:

"I think of the P2WPKH or P2WSH hash as a program itself. It's certainly an unusual one, but that hash defines the semantics entirely. Would you agree it's a program if it we extended to allow including actual opcodes in the 2-40 bytes?"

[kallewoof]

Okay, I see what you're saying. So basically, what I call witness program is the witness, and the scriptPubKey is the witness program. In fact, I may even wanna call it (witness) redeem script, since it could potentially refer to a P2SH as well, I think.

Does that match your view?

[kallerosenbaum]

Not really. I think you should simply call it "witness". That's how it's defined in BIP141. The witness in turn is (in case of p2wsh) composed of input data and a witnessScript. The witnessScript corresponds to the redeemScript in the p2sh case.

To clarify, I suggest the following rewording: "A new structure SignatureProof is added, which is a simple serializable scriptSig & witness container."

[kallewoof]

Sounds good. Thanks!

[kallewoof]

2f15277

[kallerosenbaum]

It'd help if the scriptPubKeys set was defined in the previous paragraph. Or say "... already exists in (the initially empty) scriptPubKeys set, ...". The same goes for the Verifying algorithm.

[kallewoof]

It felt obvious, but I can clarify, sure.

[kallerosenbaum]

You can't verify that the corresponding scriptPubKey matches the one given by the proof. The scriptPubKey is not in the proof.

[kallewoof]

For a funds proof, the scriptPubKey is actually derived from the transaction, and the witness program commitment (the second push in the SPK) may differ from the witness program provided in the proof. I will clarify.

[kallerosenbaum]

How do you know which proof entry belongs to this particular scriptPubKey? I suppose these algorithms require that the verifyer first gives a list of scriptPubKeys to the signer, and that list must be kept and processed in order. If not, please clarify how ordering is managed. Or should one simply test the proof entries one by one until success? Maybe you want to leave this up to the user how to handle?

[kallewoof]

Yes, the ordering is given by the verifier and preserved by the prover.

[kallerosenbaum]

Isn't it also important for the verifier to return INCONCLUSIVE for an unknown witness version?

[kallewoof]

The standard flags include a "do not allow upgradable flags" flag, which will trigger an error if a future segwit version is used (similar to how it will trigger a warning if a NOP is used).

[jl2012]

@kallewoof

I love the idea, but not sure if it's worth using an entire op code when you can probably solve it with simple message prefix and/or (as above) a custom sighash format (including scriptPubKey).

What I'm suggesting is orthogonal to your proposal. A OP_MESSAGEONLY = 0xf0 in the script means the script following the code would never be valid. For example, a scriptPubKey:

OP_IF OP_MESSAGEONLY <key_m> OP_ELSE <key_s> OP_ENDIF OP_CHECKSIG

For messaging purpose, OP_MESSAGEONLY is considered as OP_NOP and is ignored. A message could be signed with either key_m or key_s.

For spending, only key_s is valid.

I don't think it is a big problem to consume a op_code. If this is a real concern, I could modify it as follow: in message system, OP_RETURN will pop the top stack. If top stack is msg in hex, it is ignored. Otherwise, the script fails.

[kallewoof]

@prusnak

Second, there is already a way how to sign/verify messages via Segwit-in-P2SH and Native-Segwit addresses: see bitcoin/bitcoin#10542 (comment)
We are using this solution in Trezor for almost a year now and other wallets use this too already.

That seems like a good temporary workaround but I think the proposal here is a better long term solution. Your feedback on the subject is very much desired, of course!

[jimpo]

Concept ACK.

This should specify how CLTV/CSV are verified. Including an nLockTime in the SignatureProof container and an nSequence with each signature entry therein seems like it should work. Also, what is the sighash type byte on signatures produced? That should be specified.

I think the SignMessage purpose is fine as is. For the proof-of-funds purpose, constructing the sighash from a dummy, invalid transactions spending from all UTXOs that are to be signed for seems reasonable and should provide compatibility with existing hardware. As noted, the transaction should require an unspendable input to invalidate it.

[jimpo]

Why have a separator character since the sizes of the TxID + Index entries are known? If a separator character is needed, U+001e or U+001f seems like a better choice than the null byte.

[kallewoof]

Cool, I didn't know about those unicode characters. I think you're right though. I wanted to include a separator character as I do so in the other cases, but since the entries are indeed fixed size it seems pointless.

[jimpo]

Exclude the null terminator. That seems fairly C-specific.

[kallewoof]

I was going back and forth on this. By including it, you can actually "print" the string starting with "POF:". Without it, you need to sort of figure out where the string ends and the txid entries begin (e.g. take entry count * 36 and say string ends at character total len - that).

Perhaps this should work the same way the signature proof container works, i.e. a varint of the message length followed by the message sans NULL term char.

[jimpo]

I don't understand this. What tracks the state? Are the SignMessage/ProveFunds requests packaged into a single unit from which inputs is derived? Could this be stateless instead?

[kallewoof]

It's simply a set that is used to ensure that no input is given twice. It is defined once and included/updated in each verify/proof call.

For example, I try to convince you that I have X bitcoin to send you, so I send you 150 proofs of individual UTXO's. But I am sneakily including the same proof several times. I don't think this is trivial to detect.

[jimpo]

Would be helpful to see the serialization for SignMessage and ProveFunds? Or are these RPCs?

[kallewoof]

Adding serialization.

[jimpo]

What does it mean to return a "signature sig"? Is this a SignatureProof as specified above?

[kallewoof]

Yes, will clarify.

[kallewoof]

@jimpo Thanks for the valuable feedback. Unless I misunderstand what you're saying, you are suggesting that the format for a proof of funds is an actual transaction while the format for SignMessage is not. It seems like this complicates things, but this is what @prusnak is saying as well (in that he's opposed to a tx for signmessage but not for proof of funds), so perhaps it's the way to go...