Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIP 322: Generic Signed Message Format #725

Merged
merged 2 commits into from Sep 19, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.mediawiki
Expand Up @@ -840,6 +840,13 @@ Those proposing changes should consider that ultimately consent may rest with th
| BtcDrak
| Standard
| Draft
|-
| [[bip-0322.mediawiki|322]]
| Applications
| Generic Signed Message Format
| Karl-Johan Alm
| Standard
| Draft
|}

<!-- IMPORTANT! See the instructions at the top of this page, do NOT JUST add BIPs here! -->
169 changes: 169 additions & 0 deletions bip-0322.mediawiki
@@ -0,0 +1,169 @@
<pre>
BIP: 322
Layer: Applications
Title: Generic Signed Message Format
Author: Karl-Johan Alm <karljohan-alm@garage.co.jp>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
Status: Draft
Type: Standards Track
Created: 2018-09-10
License: CC0-1.0
</pre>

== Abstract ==

A standard for interoperable generic signed messages based on the Bitcoin Script format.

== Motivation ==

The current message signing standard only works for P2PKH (1...) addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already.

== Specification ==

A new structure <code>SignatureProof</code> is added, which is a simple serializable scriptSig & witnessProgram container.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the witness program was the 2-40 bytes after the witness version in the scriptPubKey. It should be the witness, or possibly witness field here, no? If so, several places from here on should change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure what you are suggesting, but the scriptPubKey is <version> <32 byte hash> where the latter is a hash of the witness program. The witness program itself is in the spending transaction's input, not in the scriptPubKey. The scriptPubKey simply commits to it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BIP141 says the hash IS the witness program. I asked about this on twitter (https://twitter.com/kallerosenbaum/status/949630404302196736) and @sipa explained it well in that thread:

"I think of the P2WPKH or P2WSH hash as a program itself. It's certainly an unusual one, but that hash defines the semantics entirely. Would you agree it's a program if it we extended to allow including actual opcodes in the 2-40 bytes?"

Copy link
Member Author

@kallewoof kallewoof Sep 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see what you're saying. So basically, what I call witness program is the witness, and the scriptPubKey is the witness program. In fact, I may even wanna call it (witness) redeem script, since it could potentially refer to a P2SH as well, I think.

Does that match your view?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. I think you should simply call it "witness". That's how it's defined in BIP141. The witness in turn is (in case of p2wsh) composed of input data and a witnessScript. The witnessScript corresponds to the redeemScript in the p2sh case.

To clarify, I suggest the following rewording: "A new structure SignatureProof is added, which is a simple serializable scriptSig & witness container."

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Two actions "Sign" and "Verify" are defined along with two *purposes* "SignMessage" and "ProveFunds".

=== SignatureProof container ===

{|class="wikitable" style="text-align: center;"
|-
!Type
!Length
!Name
!Comment
|-
|Uint32||4||flags||standard flags (1-to-1 with standard flags in Bitcoin Core)
|-
|VarInt||1-8||msglen||Number of bytes in message string, excluding NUL termination
|-
|Char*||[msglen]||msg||The message being signed for all subjects, excluding NUL termination
|-
|Uint8||1||entries||Number of proof entries<ref><strong>Why support multiple proofs?</strong> In particular with proof of funds, it is non-trivial to check a large number of individual proofs (one per UTXO) for duplicates. Software could be written to do so, but it seems more efficient to build this check into the specification itself.</ref>
|}

The above is followed by [entries] number of signature entries:

{|class="wikitable" style="text-align: center;"
|-
!Type
!Length
!Name
!Comment
|-
|VarInt||1-8||scriptsiglen||Number of bytes in scriptSig data
|-
|Uint8*||[scriptsiglen]||scriptsig||ScriptSig data
|-
|VarInt||1-8||witlen||Number of bytes in witness program data
|-
|Uint8*||[witlen]||wit||Witness program
|}

In some cases, the scriptsig or wit may be empty. If both are empty, the proof is incomplete.

=== Result Codes ===

A verification call will return a result code according to the table below.

{|class="wikitable" style="text-align: center;"
|-
!Code
!Description
|-
|INCOMPLETE||One or several of the given challenges had an empty proof. The prover may need some other entity to complete the proof.
|-
|INCONCLUSIVE||One or several of the given proofs used unknown opcodes or witness versions, perhaps due to the verifying node being outdated.
|-
|VALID||All proofs were deemed valid.
|-
|INVALID||One or more of the given proofs were invalid
|-
|SPENT||One or more of the claimed UTXO:s has been spent
|-
|ERROR||An error was encountered
|}

== Signing and Verifying ==

Let there be an empty set inputs which is populated and tested at each call to one of the actions below.

=== Purpose: SignMessage ===
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be helpful to see the serialization for SignMessage and ProveFunds? Or are these RPCs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding serialization.


The "SignMessage" purpose generates a sighash based on a scriptPubKey and a message. It emits a VALID verification result code unless otherwise stated.

# Return INVALID if scriptPubKey already exists in inputs set, otherwise insert it<ref><strong>Why track duplicates?</strong> Because a 3-entry proof is not proving 3 inputs unless they are all distinct</ref>
# Define the message pre-image as the sequence "Bitcoin Message:" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
# Let sighash = sha256(sha256(scriptPubKey || pre-image))

=== Purpose: ProveFunds ===

The "ProveFunds" purpose generates a sighash and a scriptPubKey from a transaction, an output index, and a message. For multiple simultaneous proofs, it also requires access to the ordered list of proofs. It emits a VALID verification result code unless otherwise stated.

# Let txid be the transaction ID of the transaction, and vout be the output index corresponding to the index of the output being spent
# Return INVALID if the txid:vout pair already exists in inputs set, otherwise insert it
# Return SPENT if the txid/vout is not a valid UTXO according to a Bitcoin node<ref><strong>Synced up or not?</strong> A normal verifier would use a synced up node. An auditor checking records from a client that were submitted in the past want to use a node that is synced up to the block corresponding to the proof, or the proof will fail, even if it may have been valid at the time of creation.</ref>
# Extract scriptPubKey from transaction output
# Define the message pre-image as the concatenation of the following components:<ref><strong>Why not just the UTXO data?</strong> We want the verifier to be able to challenge the prover with a custom message to sign, or anyone can reuse the POF proof for a set of UTXO:s once they have seen it, and the funds have not yet been spent</ref>
#* the string "POF:"
#* the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD), including the null terminating character (i.e. write strlen(message) + 1 bytes, for a C string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exclude the null terminator. That seems fairly C-specific.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going back and forth on this. By including it, you can actually "print" the string starting with "POF:". Without it, you need to sort of figure out where the string ends and the txid entries begin (e.g. take entry count * 36 and say string ends at character total len - that).

Perhaps this should work the same way the signature proof container works, i.e. a varint of the message length followed by the message sans NULL term char.

#* all transactions being proven for, as binary txid (little endian uint256) followed by index (little endian uint32), each separated by a single `0x00` byte
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have a separator character since the sizes of the TxID + Index entries are known? If a separator character is needed, U+001e or U+001f seems like a better choice than the null byte.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I didn't know about those unicode characters. I think you're right though. I wanted to include a separator character as I do so in the other cases, but since the entries are indeed fixed size it seems pointless.

# Let sighash = sha256(sha256(scriptPubKey || pre-image))

=== Action: Sign ===

The "Sign" action takes as input a purpose. It returns a signature or fails.

# Obtain the sighash and scriptPubKey from the purpose; FAIL if not VALID
# Derive the private key privkey for the scriptPubKey; FAIL if not VALID
# Generate and return a signature sig with privkey=privkey, sighash=sighash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does it mean to return a "signature sig"? Is this a SignatureProof as specified above?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, will clarify.


=== Action: Verify ===

The "Verify" action takes as input a standard flags value, a script sig, an optional witness program, and a purpose.
It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR.

# Obtain the sighash and scriptPubKey from the purpose; pass on result code if not VALID
# If one or more of the standard flags are unknown, return INCONCLUSIVE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it also important for the verifier to return INCONCLUSIVE for an unknown witness version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The standard flags include a "do not allow upgradable flags" flag, which will trigger an error if a future segwit version is used (similar to how it will trigger a warning if a NOP is used).

# Verify Script with flags=standard flags, scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness program, and sighash=sighash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you know which proof entry belongs to this particular scriptPubKey? I suppose these algorithms require that the verifyer first gives a list of scriptPubKeys to the signer, and that list must be kept and processed in order. If not, please clarify how ordering is managed. Or should one simply test the proof entries one by one until success? Maybe you want to leave this up to the user how to handle?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the ordering is given by the verifier and preserved by the prover.

# Return VALID if verify succeeds, otherwise return INVALID

=== Multiple Proofs ===

When more than one proof is created or verified, repeat the operation for each proof, retaining the inputs set. As noted, if the same input appears more than once, the operation must fail accordingly.

Note that the order of the entries in the proof must match the order of the entries given by the verifier.

* If any of the proofs are empty during a verification process, skip the verification and set the INCOMPLETE flag
* If a verification call returns ERROR or INVALID, return ERROR or INVALID immediately, ignoring as yet unverified entries
* After all verifications complete,
** return INCONCLUSIVE if any verification call returned INCONCLUSIVE
** return SPENT if any verification call returned SPENT
** return INCOMPLETE if the INCOMPLETE flag is set
** return VALID

== Compatibility ==

This specification is not backwards compatible with the legacy signmessage/verifymessage specification. However, legacy addresses (1...) may be used in this implementation without any problems.

== Rationale ==

<references/>

== Reference implementation ==

To do.

== Acknowledgements ==

TODO

== References ==

# Original mailing list thread: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html
# Pull request, with comments: https://github.com/bitcoin/bips/pull/725

== Copyright ==

This document is licensed under the Creative Commons CC0 1.0 Universal license.