Cannot be derived from xpublic_key

[gerag14]

Hi! How are you?
Thank you very much in advance for the work! It is an excellent library.

I report a problem:
You can only make derivations from root_xplublic_key and from root_xprivate_key. You get the error:
"PermissionError: You can't drive this master key."
But not from xpublic_key (BIP32):

Example: BIP32 Derivation Path: m / 44 '/ 0' / 0 '/ 0

as if you can see it on this site:
https://iancoleman.io/bip39/

Thank you very much again. Greetings

[meherett]

Hey @gerag14, If on your path have hardened keys, it does not work for root xpublic key, it only works for root xprivate key to drive. You can also refer #15 (comment) issue.

[gerag14]

I understand.

But if the last derivation (using from_index) is unhardened. Shouldn't it allow deriving from xpub?

I even commented on checking if you are root in the hdwallet.py file at line 487 and it works.

[meherett]

It doesn't work because it starts with 44' key, it requires root xprivate key. It also reads one by one, Sir. Use this
m/44/0/0/0 instead of this m/44'/0'/0'/0 derivation. Thanks!

[gerag14]

I shouldn't be able to bypass from an xpub if my LAST bypass was hardened. But in the path (m / 44 '/ 0' / 0 '/ 0) the last derivation is not enforced. Therefore (BIP32) should be able to derive from an xpub and the derivations would be:

m / 0/0
m / 0/1
m / 0/2

I attach the photo of the test.

Thank you in advance for all the contribution!

[meherett]

Thanks @gerag14, Attach it again please. let me see it

[alecov]

@gerag14 is correct. Unhardened key derivation works with only the xpub key (also related to issue #15) — that's the very point of derivation in this context.

As a side note, throwing a PermissionError in situations like this is completely incorrect since the meaning of this exception is related to OS-level errors (it inherits from OSError), not general purpose "you can't do it" errors.

@meherett It looks like removing lines https://github.com/meherett/python-hdwallet/blob/cd8c4f536553acbe38a9a1435380d9482ee8178b/hdwallet/hdwallet.py#L487-L488 is sufficient for this to work.

[meherett]

Hey @alecov, I think you are using this from_xpublic_key function. Derivation is not working for from_xpublic_key function. Use this from_root_xpublic_key instead of from_xpublic_key function. Just see both examples for Root XPublic Key derivations and XPublic Key https://github.com/meherett/python-hdwallet/blob/master/examples/from_root_xpublic_key.py and https://github.com/meherett/python-hdwallet/blob/master/examples/from_xpublic_key.py.

For PermissionError, I will change it into ValueError exception.

Thanks.

[alecov]

Hi @meherett. I think the issue here is still not clear to you, so let me try to explain it better.

When deriving from an unhardened path (such as m/0 or m/1/2/3/4), you do not need to have the corresponding private key. You only need the public key part. This is in accordance to BIP-32's derivation rules, quote:

The function CKDpub((Kpar, cpar), i) → (Ki, ci) computes a child extended public key from the parent extended public key. It is only defined for non-hardened child keys.

That is, the derivation function exists from xpub to xpub, no need for the corresponding parent xprv. Since the xpub encodes its exact node inside the hierarchy, it also encodes the i index above (the depth) and other details needed for derivation.

This is furtherly elaborated here:

N(m/a/b/c) = N(m/a/b)/c = N(m/a)/b/c = N(m)/a/b/c = M/a/b/c.
N(m/aH/b/c) = N(m/aH/b)/c = N(m/aH)/b/c.

Note how you can derive the xpub from eg. m/a/b, which is N(m/a/b), from N(m/a) alone.

As for your suggestion to use from_root_xpublic_key(), deriving from an arbitrary xpub with from_root_xpublic_key() does not work because this function resets the internal key structure info to be that of the root key (depth = 0, parent fingerprint = 0, etc.), instead of using the encoded material from the key. This is clearly visible from:

wallet = hdwallet.HDWallet().from_root_xpublic_key("any-non-root-xpub", strict=False)
assert wallet.xpublic_key() == wallet.root_xpublic_key() # Fails

All of this means the following must work:

wallet1 = hdwallet.HDWallet()
wallet1.from_root_xprivate_key("any-valid-xprv")
wallet1.from_path("m/1'/2'/3'") # *Any* path
xpub = wallet1.xpublic_key()

wallet2 = hdwallet.HDWallet()
wallet2.from_xpublic_key(xpub)
# The following assert must pass:
assert wallet1.xpublic_key() == wallet2.xpublic_key()

The good news is that your code is already correct in all of the above regards, excluding the forementioned check which I suggest you to alter to perhaps:

if not self._public_key \
    and not self._chain_code \
    and not self._depth \
    and not self._index \
    and not self._parent_fingerprint:
    raise ValueError("Derivation not possible without the public key.")

This allows hdwallet to create watch-only addresses derived from arbitrary xpubs.

Please let me know if the issue is clearer to you now. I can also submit you a PR with the above.

[meherett]

Hey @alecov again, Yes the issue is cleared, sir.

NOTE: In v1 of HDWallet the from_xpublic_key() function is not drivable but instead of this the from_root_xpublic_key() function is drivable.

In this code without drive, the test does not fail.

wallet = hdwallet.HDWallet().from_root_xpublic_key("any-non-root-xpub", strict=False)
assert wallet.xpublic_key() == wallet.root_xpublic_key() # Not Fails

It fails in here.

xpublic_key = "any-non-root-xpub"
wallet = hdwallet.HDWallet().from_root_xpublic_key(xpublic_key, strict=False)
assert wallet.xpublic_key() == wallet.root_xpublic_key() == xpublic_key # Fails

Because the information of your Non-Root XPublic Key changed into Root XPublic Key. Both (Non-Root XPublic Key and Root XPublic Key) have the same Public Key, Chain Code. The changes are exhibit on depth, parent_fingerprint and index values to 0.

https://github.com/meherett/python-hdwallet/blob/master/hdwallet/hdwallet.py#L279

But, It does not affect driving XPublic Keys. So, I already fixed this issue by adding extra option change_to_root=False param on from_root_xpublic_key() function.

In the upcoming version-2.0.0 of HDWallet, I will consider merging up from_root_xpublic_key() and from_xpublic_key() functions. This is because lots of comments on this issue were raised by users of this HDWallet library.

Version-2.0.0 will come up with Comand Line Interface (CLI) and I will release it in these 2 weeks.

[shdmitry2000]

can you guys explain something. I tried work with private key and i receive same error:

hdwallet = HDWallet(symbol=BTC)
xprev_key="xprv9z7iGvkix69hxkUo44zia4SM8omSDsVgiftMpfFMmnQT4ThsBy4AgfRLgSuygTHsghwoUAPxqY1JyACycDVd2KdXvCDsuoAPch9UuiCXsoq"
hdwallet.from_xprivate_key(
xprivate_key=xprev_key)

    # hdwallet.clean_derivation()
    hdwallet.from_path(path="m/44'/60'/0'/0/0")

===========>>>>>>>>>>>>>>>>>>error
File "/Users/dmitryshlymovich/workspace/PycharmProjects/test1/tests/test_hdWalletProvider.py", line 60, in test_hdwallet
hdwallet.from_path(path="m/44'/60'/0'/0/0")
File "/Users/dmitryshlymovich/workspace/PycharmProjects/test1/venv/lib/python3.8/site-packages/hdwallet/hdwallet.py", line 445, in from_path
self._derive_key_by_index(int(index[:-1]) + BIP32KEY_HARDEN)
File "/Users/dmitryshlymovich/workspace/PycharmProjects/test1/venv/lib/python3.8/site-packages/hdwallet/hdwallet.py", line 488, in _derive_key_by_index
raise PermissionError("You can't drive this master key.")
PermissionError: You can't drive this master key.

as i understand it's should work with hardening with xliv? i miss something?

[meherett]

Hey @shdmitry2000, For derivation, use this from_root_xprivate_key() function instead of from_xprivate_key() function.

NOTE: In v1 of HDWallet both from_xpublic_key() and from_xprivate_key() functions are not drivable but instead of this both from_root_xpublic_key() and from_root_xprivate_key() functions are drivable.

So, For your issue, check this one

XPRIVATE_KEY = "xprv9z7iGvkix69hxkUo44zia4SM8omSDsVgiftMpfFMmnQT4ThsBy4AgfRLgSuygTHsghwoUAPxqY1JyACycDVd2KdXvCDsuoAPch9UuiCXsoq"
hdwallet = HDWallet(symbol="BTC")
hdwallet.from_root_xprivate_key(xprivate_key=XPRIVATE_KEY, strict=False)
hdwallet.from_path(path="m/44'/60'/0'/0/0")
print(hdwallet.p2pkh_address())

Thanks!

[alecov]

Hi @meherett,

In the upcoming version-2.0.0 of HDWallet, I will consider merging up from_root_xpublic_key() and from_xpublic_key() functions. This is because lots of comments on this issue were raised by users of this HDWallet library.

This makes sense to me as well. There's little point in separating root- from non-root keys in a library, specially because xpubs/xprvs are self-contained (they contain everything needed for derivation).

Thanks for the good work.

[meherett]

Hey @alecov, It's fixed on v2.0.0 package. https://github.com/meherett/python-hdwallet/releases/tag/v2.0.0

Thanks!