Bug: the function 'deriveAddress' from a public key with ecdsa

[drhanlondon]

Hello,

Question 1: On Substrate-default (Westend testnet),

To derive an address from a publicKey with "sr25519", it works as below

import { deriveAddress } from "@substrate/txwrapper-polkadot";    

console.debug('build Keyring')
const drhan_keyring = new Keyring({ type: "sr25519"}); 

console.debug('Some mnemonic phrase, Add an account, straight mnemonic')
const drhan_substrateKeypair = drhan_keyring.addFromUri('world dutch flash motor outdoor major axis gloom rice pledge true suit'); 

console.debug(`publicKey: ${drhan_substrateKeypair.publicKey}`);

console.debug('derive account address');
const drhan_address = deriveAddress(drhan_substrateKeypair.publicKey, 42);
console.debug(`address: ${drhan_address}`);

The result is

publicKey: 214,165,91,166,6,98,81,50,52,109,162,10,132,145,143,208,63,192,99,231,139,235,101,167,8,118,60,226,18,171,165,30
address: 5Gv9E4mSjXJsicL4QAuRe8zq3MfGeTGp1cVmB24aRhSgrKqs 

But, next, to derive from a public key with "ecdsa", I did in the same way above with a different seed phrase

console.debug('build Keyring')
const shan_keyring = new Keyring({ type: "ecdsa" }); 

console.debug('Some mnemonic phrase, Add an account, straight mnemonic')
const shan_substrateKeypair = shan_keyring.addFromUri('asthma stereo budget skill frequent sunny enemy train kiwi word hold evil'); 

console.debug(`publicKey: ${shan_substrateKeypair.publicKey}`);

console.debug('derive account address');
const shan_address = deriveAddress(shan_substrateKeypair.publicKey, 42);
console.debug(`address: ${shan_address}`);

The result is

publicKey: 2,229,159,135,42,90,73,183,212,128,127,79,82,219,130,198,214,44,225,27,166,206,29,19,228,144,203,108,171,48,40,116,252
address: KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH

We can see that unexpectedly the returned value "KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH" is not an address (ss58), but just a public key (ss58).

By using Subkey tool, we can confirm that "KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH" is a public key(ss58).

#########################################################
from seed phrase

$ subkey inspect "asthma stereo budget skill frequent sunny enemy train kiwi word hold evil" --scheme Ecdsa

Secret phrase: asthma stereo budget skill frequent sunny enemy train kiwi word hold evil
Network ID: substrate
Secret seed: 0x67c9fddc7e706f33e3e20e4918e7e68abd986caade7c1d35e7596df4be9ca5bd
Public key (hex): 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc
Account ID: 0xfe8995168a62071594194772c371a72f1c57149273235556e2711f3a76dab5e0
Public key (SS58): KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH
SS58 Address: 5HpStbV2wnRhqiy8zsxeKYxJRLyKrsbH3Pyq9nrXtTeWXwcq

###################
from secret seed

$ subkey inspect 0x67c9fddc7e706f33e3e20e4918e7e68abd986caade7c1d35e7596df4be9ca5bd --scheme Ecdsa

Secret Key URI 0x67c9fddc7e706f33e3e20e4918e7e68abd986caade7c1d35e7596df4be9ca5bd is account:
Network ID: substrate
Secret seed: 0x67c9fddc7e706f33e3e20e4918e7e68abd986caade7c1d35e7596df4be9ca5bd
Public key (hex): 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc
Account ID: 0xfe8995168a62071594194772c371a72f1c57149273235556e2711f3a76dab5e0
Public key (SS58): KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH
SS58 Address: 5HpStbV2wnRhqiy8zsxeKYxJRLyKrsbH3Pyq9nrXtTeWXwcq

####################
from public key(hex)

$ subkey inspect --public 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc --scheme Ecdsa

Network ID/Version: substrate
Public key (hex): 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc
Account ID: 0xfe8995168a62071594194772c371a72f1c57149273235556e2711f3a76dab5e0
Public key (SS58): KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH
SS58 Address: KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH

#####################
from public key (ss58)
$ subkey inspect KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH --scheme Ecdsa

Public Key URI KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH is account:
Network ID/Version: substrate
Public key (hex): 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc
Account ID: 0xfe8995168a62071594194772c371a72f1c57149273235556e2711f3a76dab5e0
Public key (SS58): KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH
SS58 Address: KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH

############################

even with Substrate

$ cargo run --release -p subkey -- inspect "KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH" --scheme Ecdsa

Finished release [optimized] target(s) in 0.72s
 Running `target/release/subkey inspect KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH --scheme Ecdsa`

Public Key URI KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH is account:
Network ID/Version: substrate
Public key (hex): 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc
Account ID: 0xfe8995168a62071594194772c371a72f1c57149273235556e2711f3a76dab5e0
Public key (SS58): KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH
SS58 Address: KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH

##############################

$ cargo run --release -p subkey -- inspect "asthma stereo budget skill frequent sunny enemy train kiwi word hold evil" --scheme Ecdsa

Finished release [optimized] target(s) in 0.60s
Running target/release/subkey inspect 'asthma stereo budget skill frequent sunny enemy train kiwi word hold evil' --scheme Ecdsa
Secret phrase: asthma stereo budget skill frequent sunny enemy train kiwi word hold evil
Network ID: substrate
Secret seed: 0x67c9fddc7e706f33e3e20e4918e7e68abd986caade7c1d35e7596df4be9ca5bd
Public key (hex): 0x02e59f872a5a49b7d4807f4f52db82c6d62ce11ba6ce1d13e490cb6cab302874fc
Account ID: 0xfe8995168a62071594194772c371a72f1c57149273235556e2711f3a76dab5e0
Public key (SS58): KW87j6aqqJ8heTCTBeoxMLH1Yg2YmkeTz4Xwg4yHAmhmVX7wH
SS58 Address: 5HpStbV2wnRhqiy8zsxeKYxJRLyKrsbH3Pyq9nrXtTeWXwcq

##############################

As we see above, deriveAddress() returns a public key (ss58) in case of "ecdsa and Westend(42)" although we expect an address "5HpStbV2wnRhqiy8zsxeKYxJRLyKrsbH3Pyq9nrXtTeWXwcq"

But, this bug does not occur on Polkadot with "ecdsa"

const shan_address = deriveAddress(shan_substrateKeypair.publicKey, 0);

I would like to know whether this is a bug or not on Westend.

Question 2: as we see the above tests with Subkey and Substrate, I would like to know why "SS58 Address" does not show a proper ss58 address when inspecting with either a public key(hex) or a public key (ss58).

Thank you

[jacogr]

I cannot comment on tx-wrapper and their utils, however the deriveAddress is the @polkadot/util-crypto library is only for sr25519 types where derivation is applied. (It is only for the sr25519 specific derivation, applying soft paths to generate a new public key with the same underlying private key)

See

common/packages/util-crypto/src/address/derive.ts

Lines 17 to 23 in e5cb0ba

	/**
	* @name deriveAddress
	* @summary Creates a sr25519 derived address from the supplied and path.
	* @description
	* Creates a sr25519 derived address based on the input address/publicKey and the uri supplied.
	*/
	export function deriveAddress (who: HexString | Uint8Array | string, suri: string, ss58Format?: Prefix): string {

As for Substrate ecdsa, the 33-byte (compressed) publicKey is passed through blake2 to generate the address which is then encoded into ss58.

[drhanlondon]

Thanks for your comment.

Please let me know if my understanding below is wrong somewhere.

About: import { deriveAddress } from "@substrate/txwrapper-polkadot"
deriveAddress is defined at https://github.com/paritytech/txwrapper-core/blob/main/packages/txwrapper-core/src/core/util/deriveAddress.ts

import { encodeAddress } from '@polkadot/keyring';

/**
 * Derive an address from a cryptographic public key offline.
 *
 * @param publicKey - The public key to derive from.
 * @param ss58Format - The SS58 format to use.
 */
export function deriveAddress(
    publicKey: string | Uint8Array,
    ss58Format: number
): string {
    return encodeAddress(publicKey, ss58Format);
}

But, as you see the code above, the function deriveAddress heavily depends on the function encodeAddress(publicKey, ss58Format) which is shown on https://github.com/polkadot-js/common/blob/master/packages/util-crypto/src/address/encode.ts
This is the reason why I have left this bug report here polkadot-js/common

I will also leave this report on https://github.com/paritytech/txwrapper-core

Thanks

[jacogr]

As you can see above in the code posted by you - it encodes the publicKey as ss58. As per the keyring implementation, for ecdsa this is not the case, it follows a different path.

If you wish to re-implement the keyring pair address functionality with the lower level functions, you would need to apply the logic yourself.

For any address re-encoding, you always need to start with the address, not the publicKey. (Because anything ecdsa has hashing applied to get to the ss58 - so address decode and the recode is correct)

So in the keyring pairs, the encode functionality is the following -

common/packages/keyring/src/pair/index.ts

Lines 114 to 120 in e5cb0ba

	const encodeAddress = (): string => {
	const raw = TYPE_ADDRESS[type](publicKey);
	return type === 'ethereum'
	? ethereumEncode(raw)
	: toSS58(raw);
	};

Where the TYPE_ADDRESS applies logic based on the type -

common/packages/keyring/src/pair/index.ts

Lines 44 to 49 in e5cb0ba

	const TYPE_ADDRESS = {
	ecdsa: (p: Uint8Array) => p.length > 32 ? blake2AsU8a(p) : p,
	ed25519: (p: Uint8Array) => p,
	ethereum: (p: Uint8Array) => p.length === 20 ? p : keccakAsU8a(secp256k1Expand(p)),
	sr25519: (p: Uint8Array) => p
	};

Hence always starting from the .address of the pair and the re-encoding if you wish to change prefix. (To something different set on the keyring itself)

[TarikGul]

@jacogr Thanks for supplying the info to @drhanlondon. It's much appreciated, the code from our library was a bit of some legacy code that lacked the full support of all schemes so i fixed it up. Thanks again.

I used the TYPE_ADDRESS code you posted above since its a clean way of dealing with the schemes. That being said, i posted credit in the form of <code_below> in the following paritytech/txwrapper-core#293. Let me know if that is sufficient or if there is a more official (better) way to give credit. Thanks again :) (Also I think this is free to close as well)

/**
 * Copyright 2023 via polkadot-js/common
 * 
 * The slightly modified below logic is copyrighted from polkadot-js/common . The exact path to the code can be seen here:
 * https://github.com/polkadot-js/common/blob/e5cb0ba2b4a6b5817626cc964b4f66334f2410e4/packages/keyring/src/pair/index.ts#L44-L49
 */
const TYPE_ADDRESS = {
	ecdsa: (p: Uint8Array) => (p.length > 32 ? blake2AsU8a(p) : p),
	ed25519: (p: Uint8Array) => p,
	sr25519: (p: Uint8Array) => p,
};

[jacogr]

All good. A link is always appreciated.

[polkadot-js-bot]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.