Generated ECDH key does not match what's generated by pyelliptic

[gsalgado]

I'm trying to use coincurve on a P2P client for Ethereum. I'm starting from the pydevp2p code, which uses pyelliptic, and I just found out the ECDH keys generated by coincurve are different from those generated by pyelliptic. I was assuming I could change this pydevp2p test that ensures the generated ECDH keys match certain static values to use coincurve directly and it'd just work, but the modified version of that test does not generate the expected ECDH key

[ofek]

Thanks! I'm looking into this now. Preliminary results look ok (just quick test):

[ofek]

@gsalgado Ok, I found out what is happening!

pyelliptic uses an implementation of ECDH whereby it just returns the x coordinate of the derived point. Since there are two points that correspond to every x value, a shared secret is not a 1-1 mapping of public keys (given a specific secret key). libsecp256k1 returns a sha256 of the derived point.

From one of their devs:

[12:47:01] <andytoshi> this might trip people up in some applications, so we ensured it was not possible in our design
[12:49:34] <andytoshi> to be clear, this isn't a problem for the usual "two parties establishing a shared secret" use case, but ECDH is often used as a small part of larger cryptosystems
[12:50:14] <andytoshi> like ECDSA has a similar sign ambiguity which led to a form of witness malleability in bitcoin

[gsalgado]

@ofek well, maybe I'm missing something but that pydevp2p test seems to be checking that the ecdh key generated matches what other crypto libraries generate. In my manual testing, when generating the ecdh with coincurve I cannot authenticate with geth. Also, the ecdh generated by pyca/cryptography matches what the pydevp2p test expects: ethereum/pydevp2p#81

[gsalgado]

And here's the equivalent test for ethereum-cpp: https://github.com/ethereum/cpp-ethereum/blob/develop/test/unittests/libp2p/rlpx.cpp#L427-L429

[gsalgado]

Using the data from a similar test from geth (https://github.com/ethereum/go-ethereum/blob/5c9346/crypto/ecies/ecies_test.go#L456) also fails on coincurve

[ofek]

@gsalgado Yes, the ECDH generated by libsecp256k1 is different than what OpenSSL & others produce. Those others have an inherent malleability that the authors of libsecp256k1 chose to remove. Instead of returning x itself (remember, there can be 2!), they return a sha256 of the compressed point, i.e. sha256([02|03]|x).

[gsalgado]

But geth is using libsecp256k1, no?

[ofek]

@gsalgado Only for some operations it seems. They just do a simple point multiplication for their ECDH https://github.com/ethereum/go-ethereum/blob/5705ad004e01054c6aac173348b74c1097b40ff1/crypto/ecies/ecies.go#L120

[ofek]

https://gist.github.com/fivepiece/7155745fba94fa37ee35293983db635e