New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime/secret: add new package #21865
Comments
Related to #21374 |
Just as another example of what was mentioned at the end, even creating an AES block primitive causes an allocation right before key expansion, which shows wiping the key is inadequate, including how memguard does it. https://github.com/golang/go/blob/master/src/crypto/aes/cipher.go#L46-L47 The key expansion is a deterministic process so once This comment explains that as well. Effectively, you'd have to create custom implementations of all the existing crypto code in the standard lib, supplementary x/crypto libs, even indirect packages like In one sense, you'd have to replace |
This is going to be extremely hard to obtain as a generic guarantee, for all the reasons mentioned here, at #21374 and at awnumar/memguard#3. In particular there is no way to retroactively enforce that an interface implementation does not copy key material on the heap (at least outside of the stdlib, and we don't want a security guarantee that breaks when you use external implementations). But how about a smaller problem:
Would that solve your problem enough? |
Actually, forget |
A possibility to reliably wipe secrets used by the crypto library would be highly appreciated. For example, gocryptfs has tried to solve this problem as far as is currently possible by wiping the memory locations it has access to. However, as the crypto library creates copies of the encryption secrets by deriving encryption keys from the provided secrets it's currently not possible to completely wipe said secrets (and derived keys) from memory. |
Change https://golang.org/cl/162297 mentions this issue: |
Nothing in Go can truly guarantee a key will be gone from memory (see #21865), so remove that claim. That makes Reset useless, because unlike most Reset methods it doesn't restore the original value state, so deprecate it. Change-Id: I6bb0f7f94c7e6dd4c5ac19761bc8e5df1f9ec618 Reviewed-on: https://go-review.googlesource.com/c/162297 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change https://golang.org/cl/163438 mentions this issue: |
…t docs and deprecate it Nothing in Go can truly guarantee a key will be gone from memory (see #21865), so remove that claim. That makes Reset useless, because unlike most Reset methods it doesn't restore the original value state, so deprecate it. Change-Id: I6bb0f7f94c7e6dd4c5ac19761bc8e5df1f9ec618 Reviewed-on: https://go-review.googlesource.com/c/162297 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> (cherry picked from commit b35daca) Reviewed-on: https://go-review.googlesource.com/c/163438
Here's a somewhat different proposal that doesn't require adding anything to the language syntax itself and should deal with interesting behind-the-scenes copies: runtime.SetZeroOnGC(obj) In a similar way as |
@zx2c4 interesting solution, but since the garbage collector isn't guaranteed to run this wouldn't really provide any reasonable guarantee of secrets being erased. |
@awnumar People who must have the memory zeroed now can of course call |
@zx2c4 I understand yet it seems like considerable extra architectural effort to add this functionality to the runtime without giving it more power in the form of some kind of security guarantees. |
Is this talk of "considerable extra architectural effort" true? I was hoping one of the garbage collector maintainers authors would chime in saying something like, "oh that sounds very easy! we'll look into it." Maybe we can wait for them to weigh in on that first? And "giving it more power in the form of ..." already evaluates to |
@zx2c4 Calling Having to deal with an additional function call to ensure some security property holds adds complexity to every code path, and dealing with it can be non-trivial. |
I guess it's up to the user and the use case. If the requirement is just that "at some point in the future, it's zeroed", then don't call |
Seems like a weak security requirement. I can't think of a use-case for it. If a feature is going to be added to tackle the problem that this issue is about, it had better do it properly or not do it at all. |
Looks like you convenient ignored the following sentence, which mentions the case of a different security requirement. |
That's a strange way of looking at it. "Either we implement a perfect solution right away or we stay as insecure as we are at the moment!"
|
Well, you can zero on GC right now with a finalizer:
If/when Go has a copying GC*, then we would need to think about whether to zero the original copy of an object when it is moved. Maybe we just do that for every object. Maybe we recommend using the finalizer above, and only zero the original copy of things that have finalizers. (Kinda hacky overloading of semantics, but might work.) I think this issue is asking for something more. In particular, Go exerts no control over what the OS does with heap-backed pages. It could write them to a swap file, for example. I think in general key management is going to require separate allocations for secrets anyway (specially mapped, on secure physical memory, etc.), so adding a mechanism for Go objects is only solving part of the key management problem. *Go does have copying for stack-allocated objects. We don't currently zero the old copy of those objects. But you can force objects to be allocated off the stack by, for example, putting a finalizer on them. |
Right, sort of. My proposal is to add something that formalizes the possibility of that, via a specific call to
Having a specific marking means various facets of it can improve over time with future pull requests. For example, In other words, everyone agrees that proper zeroing in Go requires some cooperation from the runtime. Let's start that by adding a marking to the runtime. |
Speaking from experience, the ability to provide a custom allocator to the runtime or within well-defined scopes, such as within a function or within a package, would be incredibly helpful. Currently in order to use secure APIs like crypto code requires manually rewriting parts of them to use specially allocated memory. This is an evolving landscape and we're learning new things all the time about what works best and what does not. I don't personally see the value in a wrapper function for the finalizer: it adds no real new functionality to the table and it gives the impression of greater security than it provides. |
That sounds interesting. Maybe open a separate PR for that alongside some syntactical (Go 2) or function proposals? |
The problem I see with Maybe some |
Why does it matter? If you are referencing the object, it's necessarily before it's been GC'd, and so there's a chance to call |
If it is just |
Not sure I follow. For a "larger key management" situation, you call It might be the case that you'd like to redirect allocations of an entire module to some totally separate allocator or something wild. That's fine and interesting, and I hope @awnumar will open an issue with a proposal for introducing that kind of multi-allocator machinery to Golang or something. That seems like an interesting parallel effort that might benefit from being spec'd out. My proposal here is to add a simple flag to the current allocator to solve this problem in an smaller and evolutionary way that can improve overtime. |
Apologies if i've missed previous context on this: why are stack frames zeroed before secret.Do returns whereas heap allocations are only marked to be zeroed at some unspecified future time when the GC runs? |
It's a compromise between what we want for security-critical code, and what promises a GC'd language can offer. Liveness for heap values is determined by the GC, so the soonest we can safely implicitly zero a value is when the GC determines it's no longer used. For values you control directly, you can zero them explicitly at the time of your choosing, but operating on that memory might have created other heap values (e.g. in standard library functions that handled your bytes), which you can't access explicitly and can't be implicitly altered while they're alive. So, GC is the soonest those "derived" values can be cleared. OTOH, the stack is managed inline by function entry/exit code snippets, so function exit is the obvious candidate for the place where a stack frame can safely be zeroed. And since the transition we care about most is from secret mode to non-secret mode, the spec only promises stack zeroing there, to give the implementation as much freedom as possible. This doesn't provide guarantees as strong as runtime models like refcounted memory, but it still provides a good upgrade of guarantees for perfect forward secrecy: before, we had no guarantee at all that dead sensitive memory would be cleaned up. With this the runtime promises that, if the GC is running periodically, the lifetime of sensitive values won't be "much longer than needed". |
No change in consensus, so accepted. |
So this was accepted, but no work was done (since most of the effort is currently focused on (runtime/generics/compiler). Is it still planned to be done at some time in the future? |
I don’t understand the concern here. You cannot read another processed memory without root access. If you have root access you can compromise the security of the entire server - including replacing the crypto libraries with compromised versions. |
@robaho not my forte but my understanding is that it’s to protect things like the key which is only required briefly and remove/zero it from memory as quickly as possible. I assume this would minimise damage by things like spectre attacks as well as more traditional buffer overruns. Commenting mostly to see if my understanding is correct. |
The most compelling scenario here is a long-running process that is concerned with forward secrecy. If you have a Wireguard server that's been running for a year, and it gets compromised today, the protocol is designed such that it won't be possible to decrypt the past year of recorded traffic. However, if the ephemeral keys are still in memory, that property is lost. crypto/tls also uses ephemeral key exchanges and rotates session ticket keys for the same reason, but its forward secrecy properties are also weakened by the risk of having old key material remain in memory. |
Hmm. That sounds suspect to me. If that machine is compromised at a later date, it is easier to install compromised libraries and use that to access the future - and past - data that any user accessing that server has access to. If the server has ephemeral keys in memory that are of any use then the sever is acting as a “free text” gateway and compromising a free text gateway is a critical security failure. |
Forward secrecy is a widely desired, studied, and implemented property of cryptographic protocols, and such protocols should be safe to implement in Go. We are not re-debating whether forward secrecy has a reason to exist. I suggest looking up the literature on the matter. |
Final API is here: #21865 (comment)
Forward secrecy is usually only a thing if it's possible to ensure keys aren't actually in memory anymore. Other security properties, too, often require the secure erasure of keys from memory.
The typical way of doing this is through a function such as
explicit_bzero
,memzero_explicit
, or the various other functions that C library writers provide that ensure an optimization-free routine for zeroing buffers.For the most part, the same is possible in Go application code. However, it is not easily possible to do so with crypto API interfaces that internally manage a key buffer, such as the AEAD interface.
In the Go implementation of WireGuard, @rot256 has been forced to resort to unholy hacks such as:
Having to resort to this kind of reflection is a bit unsettling and something we'd much rather not do.
So, this issue is to request and track the addition of a consistent "Clear()" interface for parts of the Go crypto API that store keys in internal buffers.
Furthermore, even if real clearing is deemed to be an abject failure due to Go's GC, and one must instead mmap/mlock or use a library such as memguard, the AEAD interface still is inadequate, because it appears that SetKey winds up allocating its own buffer internally. So again, big problem.
cc: @agl
The text was updated successfully, but these errors were encountered: