runtime/secret: add new package

[zx2c4]

Final API is here: #21865 (comment)

---

Forward secrecy is usually only a thing if it's possible to ensure keys aren't actually in memory anymore. Other security properties, too, often require the secure erasure of keys from memory.

The typical way of doing this is through a function such as explicit_bzero, memzero_explicit, or the various other functions that C library writers provide that ensure an optimization-free routine for zeroing buffers.

For the most part, the same is possible in Go application code. However, it is not easily possible to do so with crypto API interfaces that internally manage a key buffer, such as the AEAD interface.

In the Go implementation of WireGuard, @rot256 has been forced to resort to unholy hacks such as:

type safeAEAD struct {
	mutex sync.RWMutex
	aead  cipher.AEAD
}

func (con *safeAEAD) clear() {
	con.mutex.Lock()
	if con.aead != nil {
		val := reflect.ValueOf(con.aead)
		elm := val.Elem()
		typ := elm.Type()
		elm.Set(reflect.Zero(typ))
		con.aead = nil
	}
	con.mutex.Unlock()
}

func (con *safeAEAD) setKey(key *[chacha20poly1305.KeySize]byte) {
	con.aead, _ = chacha20poly1305.New(key[:])
}

Having to resort to this kind of reflection is a bit unsettling and something we'd much rather not do.

So, this issue is to request and track the addition of a consistent "Clear()" interface for parts of the Go crypto API that store keys in internal buffers.

---

Furthermore, even if real clearing is deemed to be an abject failure due to Go's GC, and one must instead mmap/mlock or use a library such as memguard, the AEAD interface still is inadequate, because it appears that SetKey winds up allocating its own buffer internally. So again, big problem.

cc: @agl

[dsnet]

Related to #21374

[anitgandhi]

Just as another example of what was mentioned at the end, even creating an AES block primitive causes an allocation right before key expansion, which shows wiping the key is inadequate, including how memguard does it.

https://github.com/golang/go/blob/master/src/crypto/aes/cipher.go#L46-L47

The key expansion is a deterministic process so once c.enc and c.dec are set, if the system was compromised such that memory could be scanned, even on a system where memguard is used, an attacker could get the contents of c.enc or c.dec, reverse the key expansion process, and now they have the original key. If I'm not mistaken, you could currently call aes.NewCipher(key) where key points to a memguard buffer, immediately wipe key, then continue to use the instantiated cipher "object" for calls to Encrypt and Decrypt calls, since now only the internally expanded key is required.

This comment explains that as well. Effectively, you'd have to create custom implementations of all the existing crypto code in the standard lib, supplementary x/crypto libs, even indirect packages like math/big would end up getting into scope.

In one sense, you'd have to replace make itself to using manually managed memory, like how memguard does it, but that of course is fundamental change to Golang itself, and would make the niceties of GC pointless

[FiloSottile]

This is going to be extremely hard to obtain as a generic guarantee, for all the reasons mentioned here, at #21374 and at awnumar/memguard#3. In particular there is no way to retroactively enforce that an interface implementation does not copy key material on the heap (at least outside of the stdlib, and we don't want a security guarantee that breaks when you use external implementations).

But how about a smaller problem:

• add a Wipe() method to the implementation (say chacha20poly1305); the method might exist or not based on the platform since implementations can differ, the application can decide to warn or bail out if Wipe() is unavailable by doing an interface-upgrade from AEAD, or not to compile at all by calling it directly on the concrete type
• make an implementation with a Wipe() method guaranteed not to make copies of the key
• if necessary, add a NewWithAllocator(makeSlice func(size int) []byte) function so that key material is placed in manually allocated memory exempt from GC and swapping (for example via memguard)
  • alternatively, unsafe can be used to instantiate the chacha20poly1305 object, but then you'd need a Init() method

Would that solve your problem enough?

[FiloSottile]

Actually, forget Wipe(), it makes no sense if you have NewWithAllocator(makeSlice func(size int) []byte) since you can just use the allocator wiping feature (which memguard offers). Moreover, if you don't have neither NewWithAllocator nor a #21374 solution, Wipe() is useless because of GC copies and swapping.

[yahesh]

A possibility to reliably wipe secrets used by the crypto library would be highly appreciated. For example, gocryptfs has tried to solve this problem as far as is currently possible by wiping the memory locations it has access to.

However, as the crypto library creates copies of the encryption secrets by deriving encryption keys from the provided secrets it's currently not possible to completely wipe said secrets (and derived keys) from memory.

[gopherbot]

Change https://golang.org/cl/162297 mentions this issue: crypto/rc4: remove false guarantees from Reset docs and deprecate it

[gopherbot]

Change https://golang.org/cl/163438 mentions this issue: [release-branch.go1.12] crypto/rc4: remove false guarantees from Reset docs and deprecate it

[zx2c4]

Here's a somewhat different proposal that doesn't require adding anything to the language syntax itself and should deal with interesting behind-the-scenes copies:

runtime.SetZeroOnGC(obj)

In a similar way as SetFinalizer attaches a function to an object, SetZeroOnGC could mark that object's memory as needing to be zeroed whenever the runtime frees it or moves it to a new location. Overtime this could grow nice optimizations too that aren't as easy with manual code; for example, if an object is being GC'd in order to serve an immediate allocation request with a default initial value, then the zeroing could be skipped.

CC @mknyszek @aclements

[awnumar]

@zx2c4 interesting solution, but since the garbage collector isn't guaranteed to run this wouldn't really provide any reasonable guarantee of secrets being erased.

[zx2c4]

@awnumar People who must have the memory zeroed now can of course call runtime.GC(). People with more relaxed requirements, "memory should get zerored sometime in the future", can opt not to do that. Notably that's the same semantic that people are used to having with SetFinalizer.