Add the core implementation of a TLB-shootdown based RCU_TLB mode.
Add a new flavor of userspace RCU that uses TLB shootdowns to force readers to issue memory barriers. This works along the lines of the membarrier and signal approaches, but uses a different primitive to force threads to issue barriers: mprotect(2). When a thread revokes write permissions on a page, the kernel needs to do a TLB shootdown to make sure that none of the other CPUs running code in that address space have a writable mapping for that page cached. In Linux, this is done by forcing code to invalidate the mappings to run on every CPU in the address space, and waiting for completion. The code for the "run this function on another CPU" mechanism forces the target CPU to issue an smp_mb(). (In practice TLB shootdowns are done when permissions are added, not just when they are removed, but they needn't be; faults caused by using a cached entry with less permissions can be fixed up by the page fault handler. They're also needed when unmapping memory, but mprotect() seems cheaper than having to mmap() and munmap(). Also TLB shootdowns aren't needed if the page is non-present because it's never been backed or has been swapped out, so mlock(2) is used to keep it in place). The performance characteristics of this approach are to be roughly similar to the membarrier approach. Relative to membarrier, the mprotect()/TLB shootdown approach has one minor disadvantage, one major disadvantage, and one major advantage: * membarrier() should be slightly more efficient on the write side. The mprotect() approach needs to make two system calls instead of one and the system calls need to actually do some pointless modifications of the page table. It shouldn't be that expensive, though. * The mprotect() version is a total and complete hack. It is blatantly abusing an interface in a way that it was not designed to be used. No spec or documentation makes any guarentees about mprotect() forcing memory barriers. That said, Linux's TLB shootdown will cause every running thread in the process to call smp_mb(), and I think it is probably difficult to build an implementation that does not. * mprotect() is actually implemented in production kernels. membarrier() is clearly a better solution, but it never shipped. Relative to the signal and mb implementations, the TLB shootdown version has all of the advantages of membarrier() and continues to have the "is a total hack" disadvantage. For clarity, I split the patch up into the actual implementation and the various Makefile/test/config changes needed to build and test it; this is the former.
- Loading branch information