proposal: runtime: add support for the Arm’s ArmV8.5-A Memory Tagging Extension (MTE) in Go

[zhangfannie]

What is Memory Tagging Extension (MTE)?

It was first introduced as part of the armv8.5 instruction set in August 2019 and built into the first Armv9 compliant CPUs that were announced in May 2021. It is a security architecture feature to detect and prevent memory safety vulnerabilities before and after deployment.

Why is MTE needed for Go?

Go has structures and its memory layout is visible to the developers. We also provide pointers for accessing some memory directly. This provides a lot of flexibility to the developers and makes the interaction with C code easier. But it also opens the door for the developers to make mistakes on using the memory. So, it is necessary to enable the memory detector in Go.

We have deployed ASan, a software memory error detector. It consists of a compiler instrumentation module and a runtime library. The disadvantage is that ASan comes with a heavy performance cost (it is roughly 2x slower), which makes it unsuitable for widespread deployment. MTE can reduce this performance overhead (Its performance overhead estimate is 5% in asynchronous mode and 25% in synchronous mode) whilst offering some level of protection provides the mechanism to detect out-of-bounds and use-after-free bugs in production code with no instrumentation.

How does MTE work?

At a high level, MTE tags each memory allocation/deallocation with additional metadata. It allocates a tag to a memory location, which then can be associated with pointers that reference that memory location. At runtime the CPU checks that the pointer and the metadata tags match on each load and store.

There are two types of tagging, one is Address Tagging, it adds four bits to the lowest 4-bit of the top-byte of every pointer. Another is Memory Tagging, it also consists of four bits, linked with every aligned 16-byte region in the application’s memory space. To implement the address tagging bits without requiring larger pointers, MTE only works with 64-bit applications since it uses the Top Byte Ignore (TBI) feature, which is an feature of Arm 64bit Architecture.

MTE offers a tuneable level of performance, protection, and precision at runtime, it has 4 operating modes: OFF, Asynchronous mode, Synchronous mode, and Asymmetric mode (this is added in MTE3, for more information, see chapter D9 in Arm Architecture Reference Manual for A-profile architecture).

The details are explained in Armv8.5-A Memory Tagging Extension.

Prototype heap tagging in Go.

As we know, Glibc2.33 has support for MTE on arm64 and it just has the userspace heap tagging, see https://elixir.bootlin.com/glibc/glibc-2.33/source/sysdeps/aarch64. Glibc enables MTE using a memory tunable glibc.mem.tagging, which takes a value between 0 and 255 and acts as a bitmask that enables various capabilities, please see Memeory Related Tunables for details.

Referring to the implementation of glibc, we plan to enable MTE in go using an environment variable as well, named GOMTE, which has the same values and capabilities as glibc’s memory tunable. The GOMTE behaves as follows:

• The default value is 0, which disables all memory tagging.
• The go runtime will enable memory tagging support if GOMTE has any non-zero value.
• It is only used to control the way the Go code uses MTE feature.
• It is only supported on AArch64 systems with the MTE feature; it is ignored for all other systems.
• It cannot be used with other sanitizers (-asan, -msan or -race).
• For cgo, user needs to set glibc.mem.tagging for C code and GOMTE for Go code, respectively.
• If GOMTE is inconsistent with glibc.mem.tagging, go runtime will print a warning message and MTE is still enabled.

Note: Given the cgo situation, both GLIBC and Go runtime may configure the MTE protection level for the threads. If the configurations are inconsistent, some threads may be protected at a different level as developer expected. To avoid confusion, we prefer making GOMTE definition always aligned with glibc.mem.tagging definition. Considering that GLIBC tunables are not guaranteed to be stable, GOMTE definitions may also be changed in the future to match the behavior of GLIBC.

The implementation:

• Add CPU feature detection to check whether the hardware supports MTE.
• Add an environment variable GOMTE.
• Enable memory tagging and set tag checks fault mode based on the value of GOMTE.
• Modify the allocator when tagging is needed.
• • Allocate 16 bytes aligned pointer.
• • Tag the pointer and the memory.
• Modify the deallocator when tagging is needed.
• • Re-tag all freed memory areas with zero tagging.

Any feedback on the current design is welcome. Thank you.

[ianlancetaylor]

I'm having a hard time seeing how this can work with the garbage collector. The garbage collector must obviously be able to examine any memory location. Is the intent that the garbage collector record the tag for each page, and use that tag to construct a pointer when examining that page?

Alternatively, since Go is already a memory-safe language, should we simply reserve a memory tag for all Go memory allocations? That would let C code check its use of Go pointers, and let Go code check its use of C pointers.

[zhangfannie]

@ianlancetaylor Yes, you are right. When MTE is enabled, we should do special handling of pointers (remove and add the Address Tagging) in the process of the garbage collector. For example, a pointer should be removed its address tag when the GC checks whether it is a heap-allocated object. And when loading an underlying pointer from it, the pointer should be added its address tag.

As for the implementation, use p & 0xf0ffffffffffffff to remove the address tag of the pointer p. To add the address tag for a pointer p , we have the following three options:

1. Load the memory tag from the memory location referencedby p, and add it to p.
2. The address tag added for allocations is not randomly generated, but calculated by a hash of the start address of a span instead, so we can get the start address of the span of p, and then calculate the address tag for p.
3. Just reserve a memory/address tag for all Go memeory allocations.

These three options offer difference level of performance, protection and precision. Like, the option 1 would be incorrect if this pointer p is invalid. The option 2 has higher precision but with weaker performance. The option 3 has strong performance but with weaker protection. We are considering the best option. Thank you.