Signmessage doesn't work with segwit addresses

[karelbilek]

Describe the issue

Signmessage doesn't work on "segwit addresses" (what BIP141/BIP49 calls "P2WPKH-nested-in-P2SH").

Steps to reproduce

On testnet (or on litecoin :)):

1. NEW_ADDRESS=$(bitcoin-cli getnewaddress)
2. SEGWIT_ADDRESS=$(bitcoin-cli addwitnessaddress $NEW_ADDRESS)
3. bitcoin-cli signmessage $SEGWIT_ADDRESS "shiny"

Expected behaviour

Signature is printed

Actual behaviour

error code: -3
error message:
Address does not refer to key

What version of bitcoin-core are you using?

v0.14.1

self-compiled from github repo

Machine specs:

• OS: Linux
• CPU: can reproduce on x86 and x64 CPUs

[karelbilek]

Looking at the code, it seems to me that verification will have the exact same problem. That is,

addr.GetKeyID(keyID)

will return false.

[achow101]

The address is a p2sh address, it doesn't have an associated public or private key to sign and verify messages with.

[karelbilek]

I don't understand

It does have associated private key - the same private key as the first address I get from bitcoin-cli getnewaddress. I was under the impression that is what addwitnessaddress does (as was explained to me here - https://bitcoin.stackexchange.com/questions/52961/can-i-get-the-wallet-in-bitcoind-to-use-p2sh-segwit-on-testnet )

[achow101]

Yes, but it is still technically a P2SH address and is treated as such. The underlying script has a private key associated with it, but not the address itself.

[sipa]

The confusion here comes from the ambiguitiy in whether an address is an identifier of a key, or a shorthand for a script.

In the time when there was only one type of addresses, this was an innocent confusion to have: every address was indeed a shorthand for a P2PKH script, but also uniquely identified a private/public keypair. This is exploited in the signmessage command. It works with keys, not addresses, but uses addresses to refer to these keys.

Since P2SH, and certainly now with P2WSH/P2WPKH, this no longer works. You can't sign with an arbitrary P2SH address - even if you have the key for it - since the receiver wouldn't have the public key to verify with.

[luke-jr]

IMO it is intentional that the old sign/verify message does not work with P2SH/newer addresses. If someone wants to do message signing as a non-obsolete thing, I would recommend writing a BIP that specifies how to handle it for newer address formats, and fixes the confusion surrounding it (I have seen people try to use signed messages to prove they sent bitcoins, which the current stuff does NOT prove.)

In the meantime, perhaps verifymessage ought to throw an error rather than returning false?

[karelbilek]

You can't sign with an arbitrary P2SH address

But you can sight with your P2SH address that you have private key to, right?

And similarly, if the receiver knows that the address is "nested witness address", he can check whether a given public key belongs to that address.

I am not talking about arbitrary P2SH addresses, but really when the receiver knows it's "nested witness" address.

[achow101]

But you can sight with your P2SH address that you have private key to, right?

No, you can't.

And similarly, if the receiver knows that the address is "nested witness address", he can check whether a given public key belongs to that address.

I am not talking about arbitrary P2SH addresses, but really when the receiver knows it's "nested witness" address.

There is no way for the software that is verifying the signature to know whether the public key corresponds to a nested-p2wpkh address. It simply does not know what p2sh address it would correspond to since a public key can be a part of multiple p2sh addresses. The reason it works for p2pkh is because a public key can only correspond to one p2pkh address.

[karelbilek]

Oh. I thought (pubkey) -> (nested-p2wpkh address) function is deterministic, based on BIP 141.

[sipa]

Oh. I thought (pubkey) -> (nested-p2wpkh address) function is deterministic, based on BIP 141.

It is.

And it would in theory be possible to make signmessage work for a P2SH-P2WPKH address, in cases where the verifier knows the embedded pubkeyhash already. But in that case you don't need "sign with a witness address" functionality - you could just sign with the embedded key (see validateaddress), and have the verifier check that.

The point is to not further the misunderstanding that signmessage signs with an address - it never did. It signs with a keyhash, and verify with a keyhash.

[karelbilek]

So, in theory, the current behaviour could be "emulated" by using some different format, where the address would have to be P2SH-PWPKH and the pubkeyhash would be the part of the signature somehow.

However, that would need some new standards for the new types of addresses, as @luke-jr noted. And it's a question if it's worth it.

Got it. Thanks for the comments. Should I close the issue now?

[karelbilek]

nope I still misunderstood all the formats, sorry. It could still be possible now I think.

Since you have to recover pubkey from the signature right now, you can go to pubkeyhash and scripthash, given you somehow know the script format (in this case, p2sh-p2wpkh).

The signer would sign the message with his private key; the verifier would try that the pubkey in the signature generates the given address through p2sh, which would mean the signature is valid.

The only problem I can think of is that you are then proving both the ownership of the p2pkh and the p2s-p2wpkh addresses, but it doesn't really matter.

[gmaxwell]

We constructed a new kind of signmessage for elements which is conceptually a lot better and supports arbitrary scripts-- but it immediately runs into a problem that softfork semantics only work within the context of a consensus network... it's not clear how to handle them.

E.g. if a new softfork is defined on bitcoin and you make a pubkey that uses it... old verifiers need to return indeterminate-- before segwit script versioning this was intractable but it could be done now.

[karelbilek]

just for interest - our users (Trezor) are asking for this feature (on Litecoin addresses) and we will maybe have to write some custom format for this

see discussion here

trezor/trezor-mcu#169

[prusnak]

trezor/trezor-mcu#169

TL;DR: As a quick solution we are thinking about using introducing more v values to distinguish among various address formats:

35-38 p2sh segwit pubkey (base58)
39-42 segwit pubkey (bech32)

[karelbilek]

sign/verify for p2wpkh-in-p2sh is now implemented in trezor firmware (only in code now). See the linked discussion.

[Sjors]

New dev mailinglist discussion here.

[jakubtrnka]

I'm currently working on it

[prusnak]

@jakubtrnka working on what?

[karelbilek]

@jakubtrnka are you working on BIP?

[sipa]

@prusnak I'm totally fine with extending signmessage using just new header bytes as you're suggesting:

35-38 p2sh segwit pubkey (base58)
39-42 segwit pubkey (bech32)

As SegWit keys are always compressed you only need 4 for each, indeed.

[jakubtrnka]

@karel-3d @prusnak At first I'm going to implement verification for P2SH encapsulated P2WPKH. If that works then also "signmessage". This is not a problem. ECDSA enables to recover public key from signature as described in § 4.1.6 . is actually very similar procedure of how "normal" message/address verification works, am I right? The public key is reconstructed from signature, hashed and compared. Right now the only way I found to verify such signatures is in Trezor web wallet. I didn't find anyone else working on this.

[sipa]

I think what Luke means is this, but correct me if I'm wrong:

Signing a message with an address proves that whoever can spend money sent to that address agrees with that message.

Since currently the only supported address types are ones that correspond to a single-key policies, that indeed implies that a message signature also corresponds to "the holder of the private key corresponding to said address agrees with the message". This is however just a technicality. In BIP322 you can have message signatures from a multisig address (where just a threshold signs, and there is no "the" private key), or if you'd have an address that corresponds to script OP_TRUE, anyone would be able to "sign" for it, as anyone would be able to spend from it.

So I think the point is that in an ideal world, someone verifying a message signature shouldn't know or care about the fact that there is a single private key corresponding to that address. Due to a current implementation detail that is however still the case in practice.

[luke-jr]

It's assumed that the private key is owned by the person who does the signing.

This assumption is wrong. Third party and shared wallets are supposed to allow users to sign messages with the addresses they deposit/receive with.

[EndrII]

Any news about fix this issue?

[karelbilek]

I'm no longer involved in Bitcoin so I am not sure.

But I think the consensus here is - btc developers don't like "sign message" feature for conceptual reasons (I'm not sure if for a good reason but I don't care that much anymore :D ), they grudgingly keep it working on "legacy" addresses because of backwards compatibility, and they won't add it for new address formats.

So I don't think this will ever get fixed. I'll keep it open because some folks always come and say they want it, so, whatever.

[EndrII]

oh no, its very useful feature causerie this is only method for confirming that a person is my payer,

[sipa]

There is on-and-off work on BIP322, which should provide this function for all types of addresses (segwit, and future ones), but there isn't much progress on it right now.

[EndrII]

@sipa thanks.
I hope this will be implemented soon, as this function is very important to me at the moment.

[luke-jr]

oh no, its very useful feature causerie this is only method for confirming that a person is my payer,

That's half the point to not extending this further: This feature DOESN'T confirm a person is your payer!

[ghost]

This feature DOESN'T confirm a person is your payer!

I have read all the comments in this issue and now I am confused. I have used this in Electrum few times. One example:

Poloniex support asked me to sign message with a bitcoin address which was used for withdrawal in past. I had issues accessing my account so this was done to verify I own this Poloniex account without doing any KYC.

Signing message with a bitcoin address and verifying it is possible right now using Electrum wallet and even Joinmarket added it in JoinMarket-Org/joinmarket-clientserver#841

So what is being used when users sign the message in electrum and what is being verified? Can someone else sign a message with this bitcoin address using Electrum? tb1qr3m8k7ps0cehz6dcwz0fyhsdv9h9dlcmxusz4m If yes, then maybe signing option is useless. If no, it does prove that I own the private key for this address isn't it? Or at least I am willing to associate myself with this bitcoin address and someone can send me BTC to the address used for signing.

I understand this comment and it makes sense: #10542 (comment) and also comment by sipa that there is no public key to verify for segwit addresses.

Again two questions:

1. Then how is it working in Electrum and Joinmarket?
2. Is this a workaround? To sign messages with private key: Signmessage doesn't work with segwit addresses #10542 (comment) and verify with Signmessage doesn't work with segwit addresses #10542 (comment) ?

[sipa]

@prayank23 Presumably @luke-jr is referring to the fact that a signature with address A and message M means that whoever has control over address A (i.e. the ability to move coins sent to that address) agrees with that message. You can't use it to determine who the sender is however, because Bitcoin addresses do not have a sender address; they only have input coins and those coins may have an address they were previously under control of. In the case of a service provider that has one wallet shared by multiple users, the apparent "sender" address may belong to a different user (i.e., sending to it would credit someone else). Another example: using "sender" address signatures in a payjoin transaction would permit your joining partners to sign for you, as they too have an address that looks like a "sender".

As far as what Electrum and other software does: they're probably extending the old signmessage format to segwit addresses in an ad-hoc manner (just a bit to indicate "p2wpkh" for example). This is trivial to do, but we've been loathe to do something like this, because it doesn't scale. It cannot support multisig, and would need continuous updates to support new address schemes (e.g. Taproot) leading to a compatibility nightmare. There is a solution to this: BIP322, which defines a signmessage format that works for all address types and all future schemes. However, developmemt seems to have stalled around it.

[luke-jr]

a signature with address A and message M means that whoever has control over address A (i.e. the ability to move coins sent to that address) agrees with that message

This isn't true. The UTXOs created when address A received, may be in a shared wallet, and sent by someone entirely unrelated to address A. And this is by design - it isn't a bug in the signer.

Signed messages today prove the message is agreed on by the person who receives with address A. They cannot be used to prove anything about a sender.

[sipa]

@luke-jr Yes, I agree with what you're saying; I don't think I'm saying anything else. A signature with address A means whoever can receive on address A agrees with the message. That implies they are who will receive coins sent there, and have the ability to send them further.

It does not mean anything about transactions that look like they have "input address" A.

[ghost]

Thanks @sipa and @luke-jr

There is a solution to this: BIP322, which defines a signmessage format that works for all address types and all future schemes. However, developmemt seems to have stalled around it.

Will research more about this. I had a follow up question not sure if it's offtopic.

Can signing and verifying of messages be used for 2FA or normal authentication? Example: few darknet websites had this option to save your PGP public key when you sign up and user needs to decrypt random messages on each login which were encrypted using this key. Can bitcoin address replace PGP public key in this case? So a user will add one bitcoin address during registration. A random string is shared on each login, user needs to sign it with bitcoin address and it will be verified for successful authentication.

LNURL-AUTH is also interesting which can be used to register/login with LN wallet. Not sure about the technical details but few libraries available to test it: https://github.com/fiatjaf/awesome-lnurl#libraries and it involves signing of challenges.

[Willtech]

Key is not address is fine the address is a cryptographic construct the feature is to be able to use the address to sign and verify messages.

I remember Satoshi saying Bitcoin is better because it has chance, you could ask someone to sign a message and reveal a little bit of information.

[adiabat]

Not sure whether to open a new issue but it probably fits in here.

Maybe some day there will be a way to sign messages again. Until then, could we at least tell users that it's not supported? For example, changing the top text from "You can sign messages/agreements with your addresses" to "You can sign messages/agreements with your legacy (P2PKH) addresses"

And changing the error from "The entered address does not refer to a key." (which is incorrect, as P2WPKH refers to a key just as much as P2PKH does, and I'd say P2TR refers to a key even more!) to "Message signing for segwit addresses (addresses starting with bc1/tb1) is not supported in this version of Bitcoin Core."

It's easy enough to change the error message while it's unsupported.

This came to my attention as I was helping someone who was worried that they had lost their keys (a reasonable worry given the red error message!) - they were trying to use the sign message function on an offline computer to make sure their wallet & passwords still worked.

[Willtech]

@adiabat It is supported someone has to work on the feature and produce some programming the repository will publish meeting the consensus

Why do you think the feature was included in the earlier release?

Bitcoin is honest and I mean it must be entirely the reason signing a message was introduced was to provide some proof of an owner of an address even when they are communicating privately. The only people who don't want it are apt to try that anyone cannot provide any proof. Actually if there is proof it is better anyone can look on the public ledger to see that there is proof of a balance.

Professor. Damian A. James Williamson
Wills

[Willtech]

Also, signmessage works in V0.18.1

https://bitcoincore.org/bin/bitcoin-core-0.18.1/

@NilByte

How is this supposed to be interpreted then?

Suppose it counts as a signature if you are owner of the private key.

Professor. Damian A. James Williamson
Wills