riscv: ftrace: atmoic patching and preempt improvements #1137
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
arch_stack_walk() is called intensively in function_graph when the kernel is compiled with CONFIG_TRACE_IRQFLAGS. As a result, the kernel logs a lot of arch_stack_walk and its sub-functions into the ftrace buffer. However, these functions should not appear on the trace log because they are part of the ftrace itself. This patch references what arm64 does for the smae function. So it further prevent the re-enter kprobe issue, which is also possible on riscv. Related-to: commit 0fbcd8a ("arm64: Prohibit instrumentation on arch_stack_walk()") Fixes: 6803413 ("riscv: add CALLER_ADDRx support") Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
kernel_text_address() and __kernel_text_address() are called in arch_stack_walk() of riscv. This results in excess amount of un-related traces when the kernel is compiled with CONFIG_TRACE_IRQFLAGS. The situation worsens when function_graph is active, as it calls local_irq_save/restore in each function's entry/exit. This patch adds both functions to notrace, so they won't show up on the trace records. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Some caller-saved registers which are not defined as function arguments in the ABI can still be passed as arguments when the kernel is compiled with Clang. As a result, we must save and restore those registers to prevent ftrace from clobbering them. - [1]: https://reviews.llvm.org/D68559 Reported-by: Evgenii Shatokhin <e.shatokhin@yadro.com> Closes: https://lore.kernel.org/linux-riscv/7e7c7914-445d-426d-89a0-59a9199c45b1@yadro.com/ Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
We are changing ftrace code patching in order to remove dependency from stop_machine() and enable kernel preemption. This requires us to align functions entry at a 4-B align address. However, -falign-functions on older versions of GCC alone was not strong enoungh to align all functions. In fact, cold functions are not aligned after turning on optimizations. We consider this is a bug in GCC and turn off guess-branch-probility as a workaround to align all functions. GCC bug id: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88345 The option -fmin-function-alignment is able to align all functions properly on newer versions of gcc. So, we add a cc-option to test if the toolchain supports it. Suggested-by: Evgenii Shatokhin <e.shatokhin@yadro.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
We use an AUIPC+JALR pair to jump into a ftrace trampoline. Since instruction fetch can break down to 4 byte at a time, it is impossible to update two instructions without a race. In order to mitigate it, we initialize the patchable entry to AUIPC + NOP4. Then, the run-time code patching can change NOP4 to JALR to eable/disable ftrcae from a function. This limits the reach of each ftrace entry to +-2KB displacing from ftrace_caller. Starting from the trampoline, we add a level of indirection for it to reach ftrace caller target. Now, it loads the target address from a memory location, then perform the jump. This enable the kernel to update the target atomically. The ordering of reading/updating the targert address should be guarded by generic ftrace code, where it sends smp_rmb ipi. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Now it is safe to remove dependency from stop_machine() for us to patch code in ftrace. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Each function entry implies a call to ftrace infrastructure. And it may call into schedule in some cases. So, it is possible for preemptible kernel-mode Vector to implicitly call into schedule. Since all V-regs are caller-saved, it is possible to drop all V context when a thread voluntarily call schedule(). Besides, we currently don't pass argument through vector register, so we don't have to save/restore V-regs in ftrace trampoline. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Now, we can safely enable dynamic ftrace with kernel preemption. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
|
Upstream branch: 6d8e604 |
|
At least one diff in series https://patchwork.kernel.org/project/linux-riscv/list/?series=861494 expired. Closing PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: riscv: ftrace: atmoic patching and preempt improvements
version: 1
url: https://patchwork.kernel.org/project/linux-riscv/list/?series=861494