ECDH from compressed points with "free" sqrt

[peterdettman]

Builds on #362 as an alternative to #262.

Compressed points are moved onto an isomorphism (maybe the twist) instead of calling sqrt. After the scalar mult, parallel rsqrt/inv is used to reconstruct the correct output y-coordinate in parallel with the inversion of the z-coordinate. Total overhead vs uncompressed input is ~1%.

This implementation foregoes the jacobi check, meaning that the actual scalar multiplication could take place on a quadratic twist. This admits points with x == 0 as a possibility, but still not y == 0; the twist has a 37-bit, odd, cofactor. If this occurs, the parallel rsqrt/inv at the end will catch it and reject it. I think ECDH peers can already waste each other's time at will, so I don't think the delayed catch is a problem.

Do we know whether our formulae work "correctly" for the twist? Here we actually just need them not to crash (e.g. if an x==0 or P==INF turns up) in the scalar mult. Of course, we could always put the jacobi test back in, which has ~2% additional overhead.

Note:
The initial "decompression" produces (x,y) == (K.X, K^2) on the curve y^2 = x^3 + 7.K^3, with K != 0. If K is not a quadratic residue then this is a twist of the original curve.

[apoelstra]

I think our formulae work on the twist, but we have never checked this. In particular Pieter's sage verification code assumes b = 7. (For fun, I checked with a bunch of other b values, but only ones that differed by a factor of a sixth power, which give isomorphic curves.)

Can you get points on arbitrary twists or is there some fixed set we can check with sage?

Or maybe just checking one member of all the isomorphism classes is sufficient?

[peterdettman]

"The twist" is an isomorphism class of the curves given by all the possible non-square K values. It should suffice to check y^2 = x^3 - 7. Generally, group formulae don't make use of B (==7.K^3) and so should calculate correctly on the twist. Point validation (using B obviously) doesn't appear to crop up during a scalar mult. The derivation of secp256k1_gej_add_ge apparently involves substitution of the curve equation, so I think it bears another look (B's probably cancel out).

The x==0 possibility deserves attention; I haven't seen any problems yet. 3 divides the cofactor of the twist, and indeed {P3:P3.x == 0} is a point of order 3, with 2(P3) == -P3 (i.e. doubling negates the y coordinate).

I think though that the endomorphism breaks correctness. The twist does have an endomorphism, with of course the same 2 beta value possibilities (which depend only on the field characteristic), but the corresponding lambda values are different. In particular, multiplying a twist-point by the "normal" lambda does not preserve the y-coordinate and does not map x --> beta.x. I've generated the full endomorphism parameters for y^2 = x^3 - 7 if anyone wants them for an endomorphism-compatible test of the twisted curve.

Nevertheless, in this case we don't need a right answer (it will be discarded). Scaling a point's x-coordinate by beta still leaves a valid point (x^3 == (beta.x)^3), so group operations will all still be on valid (twist) points. The scalar decomposition is incorrect, but I can't see how that breaks anything.

[apoelstra]

The bs do cancel out in the derivation of gej_add_ge. It's used to convert between "y1^2 - y2^2" to "x1^3 - x2^3", i.e. a difference of squares in y is equivalent to a difference of cubes in x. This obviously depends on a being zero, but b is free.

As for the rest of your post I need to digest it. It's not obvious why this is the only twist we care about (or the only twist that exists?) but maybe I am just being dumb.

Edit: OK, I understand now why "nonresidue K" and "residue K" form the only two equivalence classes of curves encountered in this function. I know you know this (better than me :)), but for other readers here is an explanation that makes me happy:

1. First, if two curves y^2 = x^3 + b and y^2 = x^3 + b' over the same field have that b = k^6b' for some k, then the curves are isomorphic. (There are a few ways to derive this; it's why the effective affine trick works. IIRC the only way I've seen it derived is by purely calculational means in one of the papers linked from the effective-affine PR, but there are deeper ways I'm sure.)
2. Next, if K and K' are either both quadratic residues or both nonresidues, then their quotient is a square. This follows from multiplicity of the Legendre symbol, which indicates residueness.
3. Finally, b appears as 7K^3; so two such b values differ by a power of six iff the corresponding K values differ by a power of two iff (by (2)) the K values are either both residues or both nonresidues.

Edit2 For every cyclic subgroup of the twist, there is a new lambda value right? So in principle we can enumerate these and choose the right one for the sake of the endomorphism optimization. (Though doing this in a const-time performant way is maybe hard.) Ah! But as you say lambda doesn't matter anyway. I believe you are correct on that.

Edit3: As for as verification, I now believe that Pieter's symbolic verification is fine as-is to justify correct addition of points on the curve. The exhaustive verification needs to be amended to exhaustively check every point on a twist of the curve (which is y^2 = x^3 + 7 mod 43; I'd need to check if the twist of this has suitably interesting behaviour, e.g. having a point with x = 0, having multiple cyclic subgroups, etc)

[peterdettman]

y^2 = x^3 + 7 mod 43, generator e.g. (2,12), order 31, cofactor 1, beta 6, lambda 5.
y^2 = x^3 - 7 mod 43, generator e.g. (4,10), order 19, cofactor 3, beta 6, lambda 7. (0, 6) is a point of order 3.
Should be "suitably interesting" enough for starters. The same equation with either 67 or 79 as the field characteristic also gives cofactor 1 curves with cofactor 3 twists (the 2 points of order 3 will have x==0 of course, and the odd cofactor means no points with y==0).

[apoelstra]

Nice, thanks Peter! I'll submit a PR to add exhaustive checks for the twist using the sum of (4, 10) and (0, 6) as generator.

We've been discussing on IRC the fact that some of our internal functions are missing _var annotations, which should be fixed before 1.0 ... and that it would be nice if we could mechanically check that non-_var things only ever call non-_var things. Perhaps we want something similar for "depends on the value of b" so we could know what functions are twist-correct?

[peterdettman]

Perhaps we could flag group elements as having an implicit z factor (under VERIFY flags, similar to the way field element magnitudes are tracked) - just a flag, not the actual value. Then any functions that wouldn't work (or wouldn't work right) for such inputs can catch it. That flag can then be set wherever we use any of the isomorphism tricks, and carried through by group operations that support it.

[sipa]

Closing this because it seems incomplete, and it's not clear how much of a gain this would be in today's setting with faster inverses (also, for new use cases, an even more efficient approach like #1198 is possible).

@peterdettman Do you have any thoughts on how much this can still gain us, and if it's worth rebasing? We can reopen if need be.