crypto: import fiat-crypto implementations #40171
Comments
fiat-crypto's 64-bit P-224 implementation is about 2x as fast as Go's existing portable, constant-time P-224 implementation on amd64. Their 32-bit implementation is about the same speed on 386. I expect P-256 will be similar to Curve25519 (i.e., existing assembly implementations are faster, but still worth measuring), but using fiat-crypto for P-384 and P-521 should be both much faster and provide a constant-time implementation (unlike the current, generic math/big code). |
@agl @FiloSottile Ping. While here, I'll point out the fiat-crypto implementation also speeds up curve25519 on ppc64le:
(I don't have any ARM workstations to readily benchmark on.) |
I do have some concerns about adding new license notice requirements in the libraries, because those transitively apply to every Go binary anyone builds (that imports net/http at least). I would feel much more comfortable about this if we could get the code contributed under CLA so that the Go license notice would cover it. |
@JasonGross Do you think we can get fiat-crypto's Go code contributed under Google's CLA? The normal process is documented at https://golang.org/doc/contribute.html#cla. |
Ping @JasonGross. We'd be happy to use this code but don't want to impose new notice requirements on every Go binary. |
Ah, sorry, I meant to follow up on this earlier. As discussed on openssl/openssl#12201 (comment), MIT unfortunately doesn't permit signing CLAs on projects that it holds copyright to. :-/ |
@JasonGross, thanks for replying. I certainly understand MIT not wanting to complete CLAs. An alternative solution to our problem of imposing new notice requirements on every Go binary would be if the generator outputs could be licensed under a non-attribution license such as MIT-0 or a source-code-attribution-only license such as BSD-1-Clause. Do you think that is a possibility? |
That seems quite likely. Let me chat with me colleagues and see if it's feasible. |
Thanks very much. |
@rsc We're in the process of re-licensing under user's choice, MIT OR BSD-1-Clause. However, it seems that BSD-1-Clause is not listed under https://pkg.go.dev/license-policy, even though MIT-0 and BSD-0-Clause are. Is this an oversight? Will BSD-1-Clause in fact be sufficient? |
BSD-1-clause should be fine for our purposes. Thanks very much for tackling this. I don't know why it's not listed in pkg.go.dev. Maybe it's just not very common. CC @jba . |
Looking into it. |
BSD-1-Clause will be fine, thanks. |
Based on the discussion above, this seems like a likely accept. |
I've gotten approval from everyone and have prepared mit-plv/fiat-crypto#881. Hopefully we'll get it merged in the next couple of days. |
The code has now been relicensed under MIT OR BSD-1-Clause OR Apache-2.0 |
Thanks! |
Thanks so much @JasonGross! Accepted. |
I am kindly moving this to Go1.17, as there hasn’t been much action since October 2020. |
Since Go 1.17 development should be opening up again in the near future, it seems like a good time to discuss how to move forward here. Would it make sense for the fiat-crypto project to provide a Go module that can be vendored into the standard library, analogous to the rust crate they provide? Or is it preferred for the Go project to own running the generators and checking in the generated code somewhere? |
If the deployment of the Go module can be automated in the way the Rust crate currently is, I'm happy to merge a PR that does that (and do any additional registration steps necessary). |
@JasonGross I think it would just require creating a It would require that the code directory is buildable with One last detail is the APIs need to be exported from the package (i.e., start with an uppercase letter), at least the functionality expected to be used by end users. Conventionally, Go uses mixed caps identifiers (e.g., Happy to chat further if you have more questions. Also, @FiloSottile may have more opinions on how this should be integrated, as owner of Go's crypto packages. |
I'm nearly done with a PR to adjust the casing conventions. Two questions there: What's the convention for package names? Is the convention for type identifiers the same as for function identifiers? I'm not sure what to do about the packages, though. In particular, right now the 32-bit and 64-bit implementations have identical identifier names, but live in different file names. (This is useful, for example, in C, where you can just |
@mdempsky Could you review mit-plv/fiat-crypto#907 and let me know if the name adjustments seem good? |
@JasonGross few things that stood out to me:
|
PS: it's worthwhile to wait for @mdempsky-s comments, since he has probably a better vision how to integrate this than me. |
@JasonGross Thanks, I'll comment on that PR. |
mit-plv/fiat-crypto#907 has been merged, so things like |
@JasonGross Thanks! It looks like https://pkg.go.dev/github.com/mit-plv/fiat-crypto/fiat-go/32/curve25519 is confused about the license though. I'm guessing because the fiat-crypto repo doesn't contain any of the files mentioned in https://pkg.go.dev/license-policy. Do you think you could rename the As for vendoring, it just amounts to copying snapshots of Go module sources into the Go project's repo for redistribution. You can see the Go modules that are currently vendored at https://github.com/golang/go/tree/master/src/vendor and https://github.com/golang/go/tree/master/src/cmd/vendor. I don't think there's any action you'll need to take there. |
I'm guessing the actual issue is that the licensing files are at the project root rather than at the root of the fiat-go directory, given that https://pkg.go.dev/license-policy claims to discover licenses in |
Sounds good! Do updates to the snapshots happen automatically? On some schedule? |
Hm, good point. I also wonder if it's getting confused because of multiple licenses. I've filed #44758 to look into this. At least I don't think this should be blocking for being able to actually use fiat-go. It just prevents being able to see the documentation via the pkg.go.dev interface. I think it's a good sign that it fetched the module at all.
Sorry, I'm not sure. I think it periodically refreshes. I'm not sure how often.
They're done manually. Usually shortly before each release, and then as needed/appropriate for development between releases. Do you think that should work for fiat-go too? What's the process been like for providing code to other projects? |
And just as I post that last comment, it looks like pkg.go.dev updated, and it does recognize the license now: https://pkg.go.dev/github.com/mit-plv/fiat-crypto/fiat-go@v0.0.0-20210303142032-cc0899379641/32/curve25519 |
The fiat-crypto project (https://github.com/mit-plv/fiat-crypto) generates formally-verified, high-performance modular arithmetic implementations, useful for crypto primitives like Curve25519, Poly1305, and the NIST ECC curves that are used within the Go standard library. They're currently working on a Go backend.
BoringSSL has imported their implementations for Curve25519 and P-256: https://boringssl.googlesource.com/boringssl/+/master/third_party/fiat/
At https://go-review.googlesource.com/c/crypto/+/242177, I've uploaded a WIP CL that imports their Curve25519 implementation (w/ minor tweaks), and demonstrates a significant performance improvement over the current "generic" implementation. (The existing amd64 assembly implementation is still considerably faster though.)
This proposal is to import and make use of those implementations.
Open questions:
Which algorithms should be imported? BoringSSL only imports two. Should we import more?
Do we import both 32-bit and 64-bit implementations? We could import just one implementation and still get a performance speedup (e.g., 386 sees a -10% performance boost with curve25519_fiat_64.go, and amd64 sees a -30% boost with curve25519_fiat_32.go), but they do better still with the CPU-appropriate implementations (-30% for 386 w/ 32-bit, and -61% for amd64 w/ 64-bit).
How should the code be imported? E.g., should it be separated into a third_party or vendor directory with its own LICENSE file like how BoringSSL does it?
/cc @agl @FiloSottile
The text was updated successfully, but these errors were encountered: