mm: jit/text allocator #5150
Closed
mm: jit/text allocator #5150
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![kernel-patches-daemon-bpf[bot]](https://avatars.githubusercontent.com/u/70728002?s=60&v=4){kind=small}
|
Upstream branch: 60548b8 |
nios2 uses kmalloc() to implement module_alloc() because CALL26/PCREL26 cannot reach all of vmalloc address space. Define module space as 32MiB below the kernel base and switch nios2 to use vmalloc for module allocations. Suggested-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
module_alloc() is used everywhere as a mean to allocate memory for code. Beside being semantically wrong, this unnecessarily ties all subsystmes that need to allocate code, such as ftrace, kprobes and BPF to modules and puts the burden of code allocation to the modules code. Several architectures override module_alloc() because of various constraints where the executable memory can be located and this causes additional obstacles for improvements of code allocation. Start splitting code allocation from modules by introducing jit_text_alloc() and jit_free() APIs. Start with making jit_text_alloc() a wrapper for module_alloc() and jit_free() a replacement of module_memfree() to allow updating all call sites to use the new APIs. The name jit_text_alloc() emphasizes that the allocated memory is for executable code, the allocations of the associated data, like data sections of a module will use jit_data_alloc() interface that will be added later. Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
Several architectures override module_alloc() only to define address range for code allocations different than VMALLOC address space. Provide a generic implementation in jitalloc that uses the parameters for address space ranges, required alignment and page protections provided by architectures. The architecures must fill jit_alloc_params structure and implement jit_alloc_arch_params() that returns a pointer to that structure. This way the jitalloc initialization won't be called from every architecure, but rather from a central place, namely initialization of the core memory management. Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
…alloc Extend jitalloc parameters to accommodate more complex overrides of module_alloc() by architectures. This includes specification of a fallback range required by arm, arm64 and powerpc and support for allocation of KASAN shadow required by arm64, s390 and x86. The core implementation of jit_alloc() takes care of suppressing warnings when the initial allocation fails but there is a fallback range defined. Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
Define default parameters for address range for code allocations using the current values in module_alloc() and make jit_text_alloc() use these defaults when an architecure does not supply its specific parameters. With this, jit_text_alloc() implements memory allocation in a way compatible with module_alloc() and can be used as a replacement for module_alloc(). Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
Data related to code allocations, such as module data section, need to comply with architecture constraints for its placement and its allocation right now was done using jit_text_alloc(). Create a dedicated API for allocating data related to code allocations and allow architectures to define address ranges for data allocations. Since currently this is only relevant for powerpc variants that use the VMALLOC address space for module data allocations, automatically reuse address ranges defined for text unless address range for data is explicitly defined by an architecture. With separation of code and data allocations, data sections of the modules are now mapped as PAGE_KERNEL rather than PAGE_KERNEL_EXEC which was a default on many architectures. Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
Dynamic ftrace must allocate memory for code and this was impossible without CONFIG_MODULES. With jitalloc separated from the modules code, the jit_text_alloc() is available regardless of CONFIG_MODULE. Move jitalloc initialization to x86/mm/init.c so that it won't get compiled away when CONFIG_MODULE=n and enable dynamic ftrace unconditionally. Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
jitalloc does not depend on modules, on the contrary modules use jitalloc. To make jitalloc available when CONFIG_MODULES=n, for instance for kprobes, split jit_alloc_params initialization out from arch/kernel/module.c and compile it when CONFIG_JIT_ALLOC=y Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
kprobes depended on CONFIG_MODULES because it has to allocate memory for code. Since code allocations are now implemented with jitalloc, kprobes can be enabled in non-modular kernels. Add #ifdef CONFIG_MODULE guars for the code dealing with kprobes inside modules, make CONFIG_KPROBES select CONFIG_JIT_ALLOC and drop the dependency of CONFIG_KPROBES on CONFIG_MODULES Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
When executable memory will be allocated as ROX it won't be possible to update it using memset() and memcpy(). Introduce jit_update_copy() and jit_update_set() APIs and use them in modules loading code instead of memcpy() and memset(). Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
ftrace_process_locs sorts module mcount, which is inside RO memory. Add a ftrace_swap_func so that archs can use RO-memory-poke function to do the sorting. Signed-off-by: Song Liu <song@kernel.org>
Replace direct memory writes to memory allocated for code with text poking to allow allocation of executable memory as ROX. The only exception is arch_prepare_bpf_trampoline() that cannot jit directly into module memory yet, so it uses set_memory calls to unprotect the memory before writing to it and to protect memory in the end. Signed-off-by: Song Liu <song@kernel.org> Co-developed-by: Mike Rapoport (IBM) <rppt@kernel.org> Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
When STRICT_KERNEL_RWX or STRICT_MODULE_RWX is enabled, force text allocations to use KERNEL_PAGE_ROX. Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org>
|
Upstream branch: 60548b8 |
kernel-patches-daemon-bpf
bot
force-pushed
the
series/753026=>bpf-next
branch
from
71de5a2
to
b0d2648
Compare
|
At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=753026 irrelevant now. Closing PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: mm: jit/text allocator
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=753026