cmd/go: add fips140 module selection mechanism #70200
Comments
|
Related Issues and Documentation
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.) |
Am I understanding correctly that if |
|
How many "earlier snapshots" are kept in I wonder mainly due to thoughts of (a) distribution size with keeping many versions and (b) there may be a maintenance burden with keeping compatibility with very old modules, particularly if there are dependencies on internals in std. |
|
What is the use-case for |
|
I also wonder how a process of adding any new public API to the crypto packages would look like after this change. Before adding any new API we would have to wait for it to be available in N amount of latest fips modules (#70200 (comment))? Or is the |
AIUI submissions to the FIPS validation queue take a long time to become certified with the current NIST backlog. Some policy regimes (I think notably fedramp) may allow using modules that are in the process of becoming certified since they've had lab scrutiny if not yet final sign off. If a dependent user's requirements allow it then |
|
The idea is to keep only the latest certified and the latest in-process modules as snapshots. Sometimes we might need to have a third snapshot that is going to become in-process but isn't yet, but hopefully we can time things well enough that won't happen. Indeed, the contact surface between the standard library and the modules is a burden in both directions: the modules might use internal APIs (although I tried to minimize that) and the standard library uses the module APIs (see below).
We discussed this in abstract and we're happy with new APIs returning an error if
Indeed.
We'll maintain the
Yes. Consider that modules take up to a couple years to reach certification at the moment, so the latest certified for Go 1.26 might still be v1.24.1-fips.1. However, only the versions that are snapshotted in (Left out the x/tools/go/packages questions that I might not be the best person to answer.) |
Ok, not everything can/should return an error, but i assume panic is also fine, for new APIs, to avoid build tag we can also add a version const into the fips module. |
|
Why not double down on the pseudo module idea, and instead of having a flag at all, just use replace directives. |
|
#30241 was an accepted but unimplemented proposal for standard library vendoring, would it make sense to have the fips module work like that? |
|
Inevitably, there will come a day when the latest in-process and certified version will be found to contain a vulnerability. What -fips flag is used to update to the patched version? In OpenBSD land, we talk of versions 7.6-release (which would correspond to your certified) and 7.6-current (which is approximately your gotip) and 7.6-stable (which is the release version with only critical security and reliability patches). So another way of asking my question is what is your name for "FIPS-stable"? I got verbal assurance in public from Donna Dodson, back when she was in charge of all this, that NIST intends that even the most compliance-sensitive government clients should run "FIPS-stable" and not wait around for re-certification. It would be good to get this in writing from the current administration, but also plan in advance for version naming. There ought to be some associated public logs maintained at NIST for recording such patches. |
Good point, we discussed this but forgot to mention it in the proposal. The plan if a vulnerability is found in We can keep the previous module, too, if nonetheless we get complaints about diverging from the certified source. This can make the number of modules in |
|
Are there any values of the The interaction between the
Ping @adonovan to comment on the interaction with go/packages here. |
|
cc: @timothy-king @findleyr for x/tools implications. Seems like most of the cruft is handled by
The GODEBUG implications are also mostly internal to go list. There may be tools consequences (likely minor) of these rules:
Also, it will be a(nother) module with no go.mod file. |
No, I expect enforce to be less popular, and applications can opt-in with the regular GODEBUG mechanisms.
We've heard from a few folks that would like to make a single build and then choose to operate it in FIPS mode or not at runtime. The benefits of I think build flag and GODEBUG are pretty orthogonal: the former selects the module version, the latter whether the binary operates in FIPS mode. They only interact in that if you used the |
This aspect is import to us at Red Hat for how we build and ship our binaries. Likely, we will still have to carry a patch downstream because we prefer detection to be automatic (e.g. if the host is in FIPS mode, the binary should automatically operate in FIPS mode). However, having the mechanism be chosen at runtime from the initial design will make it easier to implement and carry this specific patch on our end. Additionally, overall it simplifies building and distributing binaries with FIPS requirements such that it won't require 2 builds for each component (FIPS / non-FIPS). |
|
This proposal has been added to the active column of the proposals project |
I am not sure I understand the motivation behind this. Why would this pseudo-module be so special compared to other modules? My naive take is that the decision to use fips (and which version) should be checked into a file like go.mod or go.work somewhere. Is there a compliance reason? |
|
@timothy-king Are you suggesting that the possible values of the @adonovan Maybe this question doesn't make sense, but if something like Bazel uses go/packages uses "go list", how do pass a |
I am not yet suggesting this. I am trying to understand why the pseudo-module selection is not more like a normal module selection. Why not write Given the proposed flag, I presume that this cannot work for some reason beyond how it is distributed/where it is fetched from. But I don't understand this reason yet. |
@timothy-king Because this is not the kind of requirement that makes sense for a single dependency or package to enforce. For example some people need to build Kubernetes with FIPS but most people don't. It doesn't make sense to put this requirement in Kubernetes's go.mod. Even among the people who want Kubernetes with FIPS, the specific version of the FIPS code (inprocess vs certified vs latest) that those people want will differ from context to context (mostly company to company), based on their chosen compliance levels. It's very much a build-time decision not a global dependency decision. (Edit: Same answer to @ianthehat's question about replace directives. We really don't want to allow the ecosystem as a whole to manipulate this setting.) |
|
|
|
If I understand correctly, you are suggesting that setting When running make.bash, not setting When not running make.bash, not setting If At run time in all cases we can set |
|
The proposal committee's main sense is just that this all seems complicated. On the other hand, based on my understanding, I think this is as complicated as necessary to support the needs of FIPS. This is also, to a significant extent, an expert interface. If you have to use FIPS mode, you're going to take the time to figure out what you're doing. |
|
Have all remaining concerns about this proposal been addressed? The proposal details are in #70200 (comment) (and the meanings of the GOFIPS140 values are in #70200 (comment)). The documentation plan is in #70200 (comment). |
Yes, and that is the default.
Yes.
Yes.
Yes.
Yes. It changes various behaviors, not the cryptographic code itself. It will mean that, as an incomplete list, (1) the init-time fips code+data hashing check runs, (2) crypto/rand.Reader runs the fips randomness generator, which seeds itself from /dev/random but then does more instead of using those bytes directly, and (3) crypto/tls changes its default permitted algorithms to be from the fips allowlist. |
I agree with this, although I think the new approach is less complicated than the module approach; thanks to @ianthehat in particular for that pushing back on that. |
|
I agree, I think reading the whole conversation it all feels very complicated, but the actual final solution is actually only moderately complicated, no more so than it needs to be, and much cleaner than some of the intermediate steps might have you believe. Thanks for taking in to account my point of view, I am happy with where we ended up! |
|
Change https://go.dev/cl/629198 mentions this issue: |
|
Change https://go.dev/cl/629196 mentions this issue: |
|
Change https://go.dev/cl/629201 mentions this issue: |
GOFIPS140 will be used to control whether to build binaries that run in FIPS-140 mode by default, as well as which version of crypto/internal/fips is used during a given build. It is a target configuration variable analogous to GOOS, GOARCH, CGO_ENABLED, and the like, so the default value is recorded in the toolchain during make.bash. This CL adds the GOFIPS140 setting to the build process and records the default for use by cmd/go. For #70200. Change-Id: Iafcb5a4207f00fae8bcd93e0184a63c72526abea Reviewed-on: https://go-review.googlesource.com/c/go/+/629196 Reviewed-by: Michael Matloob <matloob@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
GOFIPS140 does two things: (1) control whether to build binaries that run in FIPS-140 mode by default, and (2) control which version of the crypto/internal/fips source tree to use during a build. This CL implements part (1). It recognizes the GOFIPS140 settings "off" and "latest" and uses them to set the default GODEBUG=fips140 setting to "off" or "on" accordingly. The documentation for GOFIPS140 is in a follow-up CL. See cmd/go/internal/fips/fips.go for an overview. For #70200. Change-Id: I045f8ae0f19778a1e72a5cd2b6a7b0c88934fc30 Reviewed-on: https://go-review.googlesource.com/c/go/+/629198 Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
GOFIPS140 does two things: (1) control whether to build binaries that run in FIPS-140 mode by default, and (2) control which version of the crypto/internal/fips source tree to use during a build. This CL implements part (2). The older snapshot source trees are stored in GOROOT/lib/fips140 in module-formatted zip files, even though crypto/internal/fips is not technically a module. (Reusing the module packing and unpacking code avoids reinventing it.) See cmd/go/internal/fips/fips.go for an overview. The documentation for GOFIPS140 is in a follow-up CL. For #70200. Change-Id: I73a610fd2c9ff66d0cced37d51acd8053497238e Reviewed-on: https://go-review.googlesource.com/c/go/+/629201 Reviewed-by: Michael Matloob <matloob@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
|
Change https://go.dev/cl/629996 mentions this issue: |
|
Based on the discussion above, this proposal seems like a likely accept. The proposal is to add a new
Note that the definitions of inprocess and certified are specific to the Go toolchain being used, to keep builds reproducible. (That is, NIST issuing a certification for a new version does not change the meaning of Like all the other target configuration variables (for example, GOOS, GOARCH, GOARM, CC, CGO_ENABLED, ...), the GOFIPS140 setting during make.bash is "baked" into the toolchain as the default setting for builds with that toolchain. If unset, the default for standard Go toolchains will be "off". The baked-in default can be overridden by setting GOFIPS140 to a non-empty value when invoking When you run There is no user-visible API or tooling that would refer to the crypto/internal/fips code as a module. When you run While crypto/internal/fips does have a set of versions in semver syntax, they are entirely separate from the Go module system. The files in lib/fips140 will have an implementation-defined format. That format will coincidentally use the module zip and checksum file formats, but that is an implementation detail.
|
See #70200 (comment), GOFIPS140 value when building the toolchain (off when not set) is the default value for GOFIPS140, it is buildcfg.defaultGOFIPS140, export as buildcfg.DefaultGOFIPS140 that can be used in the cmd/go. For #70200 Change-Id: I5a4873a718eeefda8e65bfab51d9d3d5ad2c21b6 Reviewed-on: https://go-review.googlesource.com/c/go/+/629996 Reviewed-by: Michael Matloob <matloob@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Filippo Valsorda <filippo@golang.org>
|
No change in consensus, so accepted. 🎉 The proposal is to add a new
Note that the definitions of inprocess and certified are specific to the Go toolchain being used, to keep builds reproducible. (That is, NIST issuing a certification for a new version does not change the meaning of Like all the other target configuration variables (for example, GOOS, GOARCH, GOARM, CC, CGO_ENABLED, ...), the GOFIPS140 setting during make.bash is "baked" into the toolchain as the default setting for builds with that toolchain. If unset, the default for standard Go toolchains will be "off". The baked-in default can be overridden by setting GOFIPS140 to a non-empty value when invoking When you run There is no user-visible API or tooling that would refer to the crypto/internal/fips code as a module. When you run While crypto/internal/fips does have a set of versions in semver syntax, they are entirely separate from the Go module system. The files in lib/fips140 will have an implementation-defined format. That format will coincidentally use the module zip and checksum file formats, but that is an implementation detail.
|
aclements
changed the title
|
This is mostly done. The only thing left to do will be freezing the zip file of v1.0 once we reach source freeze with the lab, and populating inprocess.txt once the lab signs off on it. Hopefully these will both happen before Go 1.24.0, but updating inprocess.txt may also slip to a minor version. |
Important
Nov 20, 2024: The latest version of the proposal is here.
Update, Nov 11 2024: An updated version is at #70200 (comment). The change is to stop presenting the choice as anything like a module, and instead to use a GOFIPS140 environment variable, analogous to GOOS, GOARCH, CGO_ENABLED, and so on.
We are working toward getting Go crypto FIPS validated, to be able to remove BoringCrypto. The relevant code is going to be
crypto/internal/fips/.... People are going to need to be able to select between a few different crypto/internal/fips trees when using any given Go distribution. We propose to treat the crypto/internal/fips tree as its own pseudo-module, with a new go build flag,-fips140, to choose the version.In general, any build needs to choose between the actual latest source in $GOROOT/src/crypto/internal/fips and an earlier source snapshot stored in module zip form in $GOROOT/lib/fips140. (Earlier snapshots will have been through more validation processes and may be necessary for certain government-related use.)
The crypto/internal/fips module will use these version numbers:
Note that the versions order correctly: v1.24.0-fips.1 < v1.24.0. The idea is that toward the end of the 1.24.0 cycle, a snapshot would be taken and sent to a lab for validation, so that by the time 1.24.0 is released, "v1.24.0-fips.1" will be an "in-process" FIPS module, meaning a lab has validated it but NIST has not yet signed off. (When NIST signs off, it becomes a "certified" module.)
The
-fips140flag can take one of the following possible values:-fips140=off(the default) means to use the source in$GOROOT/src/crypto/internal/fipsand set the default GODEBUG for compiled binaries tofips140=off. All other flag settings set the default GODEBUG tofips140=on.-fips140=latestmeans to use the source in$GOROOT/src/crypto/internal/fips(and default binaries tofips140=on, which I will stop saying).-fips140=v1.X.Y-fips.Nmeans to use the snapshot in$GOROOT/lib/fips140/v1.X.Y-fips.N.zip.-fips140=inprocessmeans to use the version listed in$GOROOT/lib/fips140/inprocess.txt(it will bev1.X.Y-fips.Nfor some X, Y, N).-fips140=certifiedmeans to use the version listed in$GOROOT/lib/fips140/certified.txt(it too will bev1.X.Y-fips.Nfor some X, Y, N).Note that the definitions of
inprocessandcertifiedare specific to the Go toolchain being used, to keep builds reproducible. (That is, NIST issuing a certification for a new version does not change the meaning of -fips140=certified in an older Go release.)crypto/internal/fips will be like a module in the sense that:
go version -moutput with its own version, in all builds using those packages. This will enable tools like govulncheck to report vulnerabilities accurately.crypto/internal/fips will be special, or unlike a module, in these ways:
-fips140build flag is the only way to choose the version.The specialness seems unavoidable. We could go all the way and make fips completely different and not look anything like a module, but it is necessary to be able to say what version of fips a given program uses, and modules are the vocabulary we use for talking about versions. So it makes sense to reuse that vocabulary. In general the special cases seem like they will not cause much trouble. In particular, programs that use x/tools/go/packages or go list to find information about packages in a build will see a Dir set to a directory in the module cache when the alternate fips locations are being used. Since they already handle seeing other packages in the module cache, they should keep working without any changes.
The only special case that might cause trouble is if clients of module.CheckPath try to check "crypto/internal/fips". If that turns out to be a problem, we can update module.CheckPath to allow crypto/internal/fips as a special case. In my prototype of this functionality, I skipped the calls to module.CheckPath in the few places where it mattered, avoiding any changes to x/mod.
I have a prototype of this code working. It needs some cleanup and tests, but the necessary changes are quite small:
In summary, the proposal is to establish the crypto/internal/fips pseudo-module with the version scheme and properties described above, and to add the
-fips140go build flag. (Note that “build flags” apply not just togo buildbut to most other go commands, includinggo install,go test, andgo list.)The text was updated successfully, but these errors were encountered: