Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
mm, slab: add static key for should_failslab()
Since commit 4f6923f ("mm: make should_failslab always available for fault injection") should_failslab() is unconditionally a noinline function. This adds visible overhead to the slab allocation hotpath, even if the function is empty. With CONFIG_FAILSLAB=y there's additional overhead, even when the functionality is not activated by a boot parameter or via debugfs. The overhead can be eliminated with a static key around the callsite. Fault injection and error injection frameworks including bpf can now be told that this function has a static key associated, and are able to enable and disable it accordingly. Additionally, compile out all relevant code if neither CONFIG_FAILSLAB nor CONFIG_FUNCTION_ERROR_INJECTION is enabled. When only the latter is not enabled, make should_failslab() static inline instead of noinline. To demonstrate the reduced overhead of calling an empty should_failslab() function, a kernel build with CONFIG_FUNCTION_ERROR_INJECTION enabled but CONFIG_FAILSLAB disabled, and CPU mitigations enabled, was used in a qemu-kvm (virtme-ng) on AMD Ryzen 7 2700 machine, and execution of a program trying to open() a non-existent file was measured 3 times: for (int i = 0; i < 10000000; i++) { open("non_existent", O_RDONLY); } After this patch, the measured real time was 4.3% smaller. Using perf profiling it was verified that should_failslab was gone from the profile. With CONFIG_FAILSLAB also enabled, the patched kernel performace was unaffected, as expected, while unpatched kernel's performance was worse, resulting in the relative speedup being 10.5%. This means it no longer needs to be an option suitable only for debug kernel builds. Acked-by: Alexei Starovoitov <ast@kernel.org> Reviewed-by: Roman Gushchin <roman.gushchin@linux.dev> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
- Loading branch information