Skip to content
Permalink
Masami-Hiramat…
Switch branches/tags

Commits on Jan 24, 2022

  1. docs: fprobe: Add fprobe description to ftrace-use.rst

    Add a documentation of fprobe for the user who needs
    this interface.
    
    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  2. fprobe: Add sample program for fprobe

    Add a sample program for the fprobe.
    
    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  3. fprobe: Add exit_handler support

    Add exit_handler to fprobe. fprobe + rethook allows us
    to hook the kernel function return without fgraph tracer.
    Eventually, the fgraph tracer will be generic array based
    return hooking and fprobe may use it if user requests.
    Since both array-based approach and list-based approach
    have Pros and Cons, (e.g. memory consumption v.s. less
    missing events) it is better to keep both but fprobe
    will provide the same exit-handler interface.
    
    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  4. arm64: rethook: Add arm64 rethook implementation

    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  5. ARM: rethook: Add rethook arm implementation

    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  6. rethook: x86: Add rethook x86 implementation

    Add rethook for x86 implementation. Most of the code
    has been copied from kretprobes on x86.
    
    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  7. rethook: Add a generic return hook

    Add a return hook framework which hooks the function
    return. Most of the idea came from the kretprobe, but
    this is independent from kretprobe.
    Note that this is expected to be used with other
    function entry hooking feature, like ftrace, fprobe,
    adn kprobes. Eventually this will replace the
    kretprobe (e.g. kprobe + rethook = kretprobe), but
    at this moment, this is just a additional hook.
    
    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  8. fprobe: Add ftrace based probe APIs

    The fprobe is a wrapper API for ftrace function tracer.
    Unlike kprobes, this probes only supports the function entry, but
    it can probe multiple functions by one fprobe. The usage is almost
    same as the kprobe, user will specify the function names by
    fprobe::syms, the number of syms by fprobe::nentry,
    and the user handler by fprobe::entry_handler.
    
    struct fprobe fp = { 0 };
    const char *targets[] = { "func1", "func2", "func3"};
    
    fp.handler = user_handler;
    fp.nentry = ARRAY_SIZE(targets);
    fp.syms = targets;
    
    ret = register_fprobe(&fp);
    
    CAUTION: if user entry handler changes registers including
    ip address, it will be applied when returns from the
    entry handler. So user handler must recover it.
    
    Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
    mhiramat authored and intel-lab-lkp committed Jan 24, 2022
  9. ftrace: Add ftrace_set_filter_ips function

    Adding ftrace_set_filter_ips function to be able to set filter on
    multiple ip addresses at once.
    
    With the kprobe multi attach interface we have cases where we need to
    initialize ftrace_ops object with thousands of functions, so having
    single function diving into ftrace_hash_move_and_update_ops with
    ftrace_lock is faster.
    
    The functions ips are passed as unsigned long array with count.
    
    Signed-off-by: Jiri Olsa <jolsa@kernel.org>
    Jiri Olsa authored and intel-lab-lkp committed Jan 24, 2022

Commits on Jan 23, 2022

  1. ftrace: Fix assuming build time sort works for s390

    To speed up the boot process, as mcount_loc needs to be sorted for ftrace
    to work properly, sorting it at build time is more efficient than boot up
    and can save milliseconds of time. Unfortunately, this change broke s390
    as it will modify the mcount_loc location after the sorting takes place
    and will put back the unsorted locations. Since the sorting is skipped at
    boot up if it is believed that it was sorted at run time, ftrace can crash
    as its algorithms are dependent on the list being sorted.
    
    Add a new config BUILDTIME_MCOUNT_SORT that is set when
    BUILDTIME_TABLE_SORT but not if S390 is set. Use this config to determine
    if sorting should take place at boot up.
    
    Link: https://lore.kernel.org/all/yt9dee51ctfn.fsf@linux.ibm.com/
    
    Fixes: 72b3942 ("scripts: ftrace - move the sort-processing in ftrace_init")
    Reported-by: Sven Schnelle <svens@linux.ibm.com>
    Tested-by: Heiko Carstens <hca@linux.ibm.com>
    Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
    rostedt committed Jan 23, 2022

Commits on Jan 18, 2022

  1. script/sorttable: Fix some initialization problems

    elf_mcount_loc and mcount_sort_thread definitions are not
    initialized immediately within the function, which can cause
    the judgment logic to use uninitialized values when the
    initialization logic of subsequent code fails.
    
    Link: https://lkml.kernel.org/r/20211212113358.34208-2-yinan@linux.alibaba.com
    Link: https://lkml.kernel.org/r/20220118065241.42364-1-yinan@linux.alibaba.com
    
    Fixes: 72b3942 ("scripts: ftrace - move the sort-processing in ftrace_init")
    Tested-by: Nathan Chancellor <nathan@kernel.org>
    Reviewed-by: Nathan Chancellor <nathan@kernel.org>
    Signed-off-by: Yinan Liu <yinan@linux.alibaba.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Yinan Liu authored and rostedt committed Jan 18, 2022

Commits on Jan 14, 2022

  1. tracing: Add ustring operation to filtering string pointers

    Since referencing user space pointers is special, if the user wants to
    filter on a field that is a pointer to user space, then they need to
    specify it.
    
    Add a ".ustring" attribute to the field name for filters to state that the
    field is pointing to user space such that the kernel can take the
    appropriate action to read that pointer.
    
    Link: https://lore.kernel.org/all/yt9d8rvmt2jq.fsf@linux.ibm.com/
    
    Fixes: 77360f9 ("tracing: Add test for user space strings when filtering on string pointers")
    Tested-by: Sven Schnelle <svens@linux.ibm.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    rostedt committed Jan 14, 2022

Commits on Jan 13, 2022

  1. rtla: Add rtla timerlat hist documentation

    Man page for rtla timerlat hist mode.
    
    Link: https://lkml.kernel.org/r/4a184003fdb81e23be3fe5ec882b1c89d5a95458.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  2. rtla: Add rtla timerlat top documentation

    Man page for rtla timerlat top mode.
    
    Link: https://lkml.kernel.org/r/58c3d6212e6c6f1f012deb2e998dd082da92075f.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  3. rtla: Add rtla timerlat documentation

    Man page for rtla timerlat tool.
    
    Link: https://lkml.kernel.org/r/78678b8d024bf5a3a79f831ac9441b96e8d2f56e.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  4. rtla: Add rtla osnoise hist documentation

    Man page for rtla osnoise hist mode.
    
    Link: https://lkml.kernel.org/r/d43cc5d516b8db180575c4b66c4aa67c6a724a8d.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  5. rtla: Add rtla osnoise top documentation

    Man page for rtla osnoise top mode.
    
    Link: https://lkml.kernel.org/r/445aa2173ca152fc2e68719e3c1a2547dd01efd3.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  6. rtla: Add rtla osnoise man page

    Man page for rtla osnoise command.
    
    Link: https://lkml.kernel.org/r/304747c602e46f6396e3bb75dfdcb42ae3656dca.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  7. rtla: Add Documentation

    Adds the basis for rtla documentation. This patch also
    includes the rtla(1) man page.
    
    As suggested by Jonathan Corbet, we are placing these man
    pages at Documentation/tools/rtla, using rst format. It
    is not linked to the official documentation, though.
    
    The Makefile is based on bpftool's Documentation one.
    
    Link: https://lkml.kernel.org/r/5f510f3e962fc0cd531c43f5a815544dd720c3f2.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Suggested-by: Steven Rostedt <rostedt@goodmis.org>
    Suggested-by: Jonathan Corbet <corbet@lwn.net>
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  8. rtla/timerlat: Add timerlat hist mode

    The rtla hist hist mode displays a histogram of each tracer event
    occurrence, both for IRQ and timer latencies. The tool also allows
    many configurations of the timerlat tracer and the collection of
    the tracer output.
    
    Here is one example of the rtla timerlat hist mode output:
      ---------- %< ----------
     [root@alien ~]# rtla timerlat hist -c 0-3 -d 1M
     # RTLA timerlat histogram
     # Time unit is microseconds (us)
     # Duration:   0 00:01:00
     Index   IRQ-000   Thr-000   IRQ-001   Thr-001   IRQ-002   Thr-002   IRQ-003   Thr-003
     0         58572         0     59373         0     58691         0     58895         0
     1          1422     57021       628     57241      1310     56160      1102     56805
     2             6      2931         0      2695         0      3567         4      3031
     3             1        40         0        53         0       260         0       142
     4             0         7         0         5         0         6         0        17
     5             0         2         0         5         0         7         0         4
     6             0         0         0         2         0         1         0         1
     8             0         0         0         0         0         0         0         1
     over:         0         0         0         0         0         0         0         0
     count:    60001     60001     60001     60001     60001     60001     60001     60001
     min:          0         1         0         1         0         1         0         1
     avg:          0         1         0         1         0         1         0         1
     max:          3         5         1         6         1         6         2         8
      ---------- >% ----------
    
    Running
     - rtla timerlat hist --help
    provides information about the available options.
    
    Link: https://lkml.kernel.org/r/7049ed3c46b7d6aceab18ffe7770003dfc4ddceb.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  9. rtla: Add timerlat tool and timelart top mode

    The rtla timerlat tool is an interface for the timerlat tracer.
    The timerlat tracer dispatches a kernel thread per-cpu. These threads set a
    periodic timer to wake themselves up and go back to sleep. After the
    wakeup, they collect and generate useful information for the debugging of
    operating system timer latency.
    
    The timerlat tracer outputs information in two ways. It periodically
    prints the timer latency at the timer IRQ handler and the Thread handler.
    It also provides information for each noise via the osnoise tracepoints.
    
    The rtla timerlat top mode displays a summary of the periodic output from
    the timerlat tracer.
    
    Here is one example of the rtla timerlat tool output:
     ---------- %< ----------
    [root@alien ~]# rtla timerlat top -c 0-3 -d 1m
                                         Timer Latency
      0 00:01:00   |          IRQ Timer Latency (us)        |         Thread Timer Latency (us)
    CPU COUNT      |      cur       min       avg       max |      cur       min       avg       max
      0 #60001     |        0         0         0         3 |        1         1         1         6
      1 #60001     |        0         0         0         3 |        2         1         1         5
      2 #60001     |        0         0         1         6 |        1         1         2         7
      3 #60001     |        0         0         0         7 |        1         1         1        11
     ---------- >% ----------
    
    Running:
      # rtla timerlat --help
      # rtla timerlat top --help
    provides information about the available options.
    
    Link: https://lkml.kernel.org/r/e95032e20c2b88c962195bf7693bb53c9ebcced8.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  10. rtla/osnoise: Add the hist mode

    The rtla osnoise hist tool collects all osnoise:sample_threshold
    occurrence in a histogram, displaying the results in a user-friendly
    way. The tool also allows many configurations of the osnoise tracer
    and the collection of the tracer output.
    
    Here is one example of the rtla osnoise hist tool output:
      ---------- %< ----------
     [root@f34 ~]# rtla osnoise hist --bucket-size 10 --entries 100 -c 0-8 -d 1M -r 9000 -P F:1
     # RTLA osnoise histogram
     # Time unit is microseconds (us)
     # Duration:   0 00:01:00
     Index   CPU-000   CPU-001   CPU-002   CPU-003   CPU-004   CPU-005   CPU-006   CPU-007   CPU-008
     0           430       434       352       455       440       463       467       436       484
     10           88        88        92       141       120       100       126       166       100
     20           19         7        12        22         8         8        13        13        16
     30            6         0         2         0         1         2         2         1         0
     50            0         0         0         0         0         0         1         0         0
     over:         0         0         0         0         0         0         0         0         0
     count:      543       529       458       618       569       573       609       616       600
     min:          0         0         0         0         0         0         0         0         0
     avg:          0         0         0         0         0         0         0         0         0
     max:         30        20        30        20        30        30        50        30        20
      ---------- >% ----------
    
    Running
     - rtla osnoise hist --help
    
    provides information about the available options.
    
    Link: https://lkml.kernel.org/r/c68060544de89b8b62510ed91c7369f162eb465b.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  11. rtla/osnoise: Add osnoise top mode

    The rtla osnoise tool is an interface for the osnoise tracer. The
    osnoise tracer dispatches a kernel thread per-cpu. These threads read
    the time in a loop while with preemption, softirqs and IRQs enabled,
    thus allowing all the sources of osnoise during its execution. The
    osnoise threads take note of the entry and exit point of any source
    of interferences, increasing a per-cpu interference counter. The
    osnoise tracer also saves an interference counter for each source
    of interference.
    
    The rtla osnoise top mode displays information about the periodic
    summary from the osnoise tracer.
    
    One example of rtla osnoise top output is:
    
    [root@alien ~]# rtla osnoise top -c 0-3 -d 1m -q -r 900000 -P F:1
                                             Operating System Noise
    duration:   0 00:01:00 | time is in us
    CPU Period       Runtime        Noise  % CPU Aval   Max Noise   Max Single          HW          NMI          IRQ      Softirq       Thread
      0 #58         52200000         1031    99.99802          91           60           0            0        52285            0          101
      1 torvalds#59         53100000            5    99.99999           5            5           0            9        53122            0           18
      2 torvalds#59         53100000            7    99.99998           7            7           0            8        53115            0           18
      3 torvalds#59         53100000         8274    99.98441         277           23           0            9        53778            0          660
    
    "rtla osnoise top --help" works and provide information about the
    available options.
    
    Link: https://lkml.kernel.org/r/0d796993abf587ae5a170bb8415c49368d4999e1.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  12. rtla: Add osnoise tool

    The osnoise tool is the interface for the osnoise tracer. The osnoise
    tool will have multiple "modes" with different outputs. At this point,
    no mode is included.
    
    The osnoise.c includes the osnoise_context abstraction. It serves to
    read-save-change-restore the default values from tracing/osnoise/
    directory. When the context is deleted, the default values are restored.
    
    It also includes some other helper functions for managing osnoise
    tracer sessions.
    
    With these bits and pieces in place, we can start adding some
    functionality to rtla.
    
    Link: https://lkml.kernel.org/r/2d44c21ff561f503b4c7b1813892761818118460.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  13. rtla: Helper functions for rtla

    This is a set of utils and tracer helper functions. They are used by
    rtla mostly to parse config, display data and some trace operations that
    are not part of libtracefs (because they are only useful it for this
    case).
    
    Link: https://lkml.kernel.org/r/a94c128aba9e6e66d502b7094f2e8c7ac95b12e5.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  14. rtla: Real-Time Linux Analysis tool

    The rtla is a meta-tool that includes a set of commands that aims
    to analyze the real-time properties of Linux. But instead of testing
    Linux as a black box, rtla leverages kernel tracing capabilities to
    provide precise information about the properties and root causes of
    unexpected results.
    
    rtla --help works and provide information about the available options.
    
    This is just the "main" and the Makefile, no function yet.
    
    Link: https://lkml.kernel.org/r/bf9118ed43a09e6c054c9a491cbe7411ad1acd89.1639158831.git.bristot@kernel.org
    
    Cc: Tao Zhou <tao.zhou@linux.dev>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Juri Lelli <juri.lelli@redhat.com>
    Cc: Clark Williams <williams@redhat.com>
    Cc: John Kacur <jkacur@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
    Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
    Cc: linux-rt-users@vger.kernel.org
    Cc: linux-trace-devel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Daniel Bristot de Oliveira authored and rostedt committed Jan 13, 2022
  15. tracing/osnoise: Properly unhook events if start_per_cpu_kthreads() f…

    …ails
    
    If start_per_cpu_kthreads() called from osnoise_workload_start() returns
    error, event hooks are left in broken state: unhook_irq_events() called
    but unhook_thread_events() and unhook_softirq_events() not called, and
    trace_osnoise_callback_enabled flag not cleared.
    
    On the next tracer enable, hooks get not installed due to
    trace_osnoise_callback_enabled flag.
    
    And on the further tracer disable an attempt to remove non-installed
    hooks happened, hitting a WARN_ON_ONCE() in tracepoint_remove_func().
    
    Fix the error path by adding the missing part of cleanup.
    While at this, introduce osnoise_unhook_events() to avoid code
    duplication between this error path and normal tracer disable.
    
    Link: https://lkml.kernel.org/r/20220109153459.3701773-1-nikita.yushchenko@virtuozzo.com
    
    Cc: stable@vger.kernel.org
    Fixes: bce29ac ("trace: Add osnoise tracer")
    Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org>
    Signed-off-by: Nikita Yushchenko <nikita.yushchenko@virtuozzo.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Nikita Yushchenko authored and rostedt committed Jan 13, 2022
  16. tracing: Remove duplicate warnings when calling trace_create_file()

    Since the same warning message is already printed in the
    trace_create_file() function, there is no need to print it again.
    
    Link: https://lkml.kernel.org/r/20220109162232.361747-1-ytcoode@gmail.com
    
    Signed-off-by: Yuntao Wang <ytcoode@gmail.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    wangyuntao authored and rostedt committed Jan 13, 2022
  17. tracing/kprobes: 'nmissed' not showed correctly for kretprobe

    The 'nmissed' column of the 'kprobe_profile' file for kretprobe is
    not showed correctly, kretprobe can be skipped by two reasons,
    shortage of kretprobe_instance which is counted by tk->rp.nmissed,
    and kprobe itself is missed by some reason, so to show the sum.
    
    Link: https://lkml.kernel.org/r/20220107150242.5019-1-xyz.sun.ok@gmail.com
    
    Cc: stable@vger.kernel.org
    Fixes: 4a846b4 ("tracing/kprobes: Cleanup kprobe tracer code")
    Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
    Signed-off-by: Xiangyang Zhang <xyz.sun.ok@gmail.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    shellbombs authored and rostedt committed Jan 13, 2022
  18. tracing: Add test for user space strings when filtering on string poi…

    …nters
    
    Pingfan reported that the following causes a fault:
    
      echo "filename ~ \"cpu\"" > events/syscalls/sys_enter_openat/filter
      echo 1 > events/syscalls/sys_enter_at/enable
    
    The reason is that trace event filter treats the user space pointer
    defined by "filename" as a normal pointer to compare against the "cpu"
    string. The following bug happened:
    
     kvm-03-guest16 login: [72198.026181] BUG: unable to handle page fault for address: 00007fffaae8ef60
     #PF: supervisor read access in kernel mode
     #PF: error_code(0x0001) - permissions violation
     PGD 80000001008b7067 P4D 80000001008b7067 PUD 2393f1067 PMD 2393ec067 PTE 8000000108f47867
     Oops: 0001 [#1] PREEMPT SMP PTI
     CPU: 1 PID: 1 Comm: systemd Kdump: loaded Not tainted 5.14.0-32.el9.x86_64 #1
     Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
     RIP: 0010:strlen+0x0/0x20
     Code: 48 89 f9 74 09 48 83 c1 01 80 39 00 75 f7 31 d2 44 0f b6 04 16 44 88 04 11
           48 83 c2 01 45 84 c0 75 ee c3 0f 1f 80 00 00 00 00 <80> 3f 00 74 10 48 89 f8
           48 83 c0 01 80 38 00 75 f7 48 29 f8 c3 31
     RSP: 0018:ffffb5b900013e48 EFLAGS: 00010246
     RAX: 0000000000000018 RBX: ffff8fc1c49ede00 RCX: 0000000000000000
     RDX: 0000000000000020 RSI: ffff8fc1c02d601c RDI: 00007fffaae8ef60
     RBP: 00007fffaae8ef60 R08: 0005034f4ddb8ea4 R09: 0000000000000000
     R10: ffff8fc1c02d601c R11: 0000000000000000 R12: ffff8fc1c8a6e380
     R13: 0000000000000000 R14: ffff8fc1c02d6010 R15: ffff8fc1c00453c0
     FS:  00007fa86123db40(0000) GS:ffff8fc2ffd00000(0000) knlGS:0000000000000000
     CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
     CR2: 00007fffaae8ef60 CR3: 0000000102880001 CR4: 00000000007706e0
     DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
     DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
     PKRU: 55555554
     Call Trace:
      filter_pred_pchar+0x18/0x40
      filter_match_preds+0x31/0x70
      ftrace_syscall_enter+0x27a/0x2c0
      syscall_trace_enter.constprop.0+0x1aa/0x1d0
      do_syscall_64+0x16/0x90
      entry_SYSCALL_64_after_hwframe+0x44/0xae
     RIP: 0033:0x7fa861d88664
    
    The above happened because the kernel tried to access user space directly
    and triggered a "supervisor read access in kernel mode" fault. Worse yet,
    the memory could not even be loaded yet, and a SEGFAULT could happen as
    well. This could be true for kernel space accessing as well.
    
    To be even more robust, test both kernel and user space strings. If the
    string fails to read, then simply have the filter fail.
    
    Note, TASK_SIZE is used to determine if the pointer is user or kernel space
    and the appropriate strncpy_from_kernel/user_nofault() function is used to
    copy the memory. For some architectures, the compare to TASK_SIZE may always
    pick user space or kernel space. If it gets it wrong, the only thing is that
    the filter will fail to match. In the future, this needs to be fixed to have
    the event denote which should be used. But failing a filter is much better
    than panicing the machine, and that can be solved later.
    
    Link: https://lore.kernel.org/all/20220107044951.22080-1-kernelfans@gmail.com/
    Link: https://lkml.kernel.org/r/20220110115532.536088fd@gandalf.local.home
    
    Cc: stable@vger.kernel.org
    Cc: Ingo Molnar <mingo@kernel.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Masami Hiramatsu <mhiramat@kernel.org>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Reported-by: Pingfan Liu <kernelfans@gmail.com>
    Tested-by: Pingfan Liu <kernelfans@gmail.com>
    Fixes: 87a342f ("tracing/filters: Support filtering for char * strings")
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    rostedt committed Jan 13, 2022
  19. tracing: Have syscall trace events use trace_event_buffer_lock_reserve()

    Currently, the syscall trace events call trace_buffer_lock_reserve()
    directly, which means that it misses out on some of the filtering
    optimizations provided by the helper function
    trace_event_buffer_lock_reserve(). Have the syscall trace events call that
    instead, as it was missed when adding the update to use the temp buffer
    when filtering.
    
    Link: https://lkml.kernel.org/r/20220107225839.823118570@goodmis.org
    
    Cc: stable@vger.kernel.org
    Cc: Ingo Molnar <mingo@kernel.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Tom Zanussi <zanussi@kernel.org>
    Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
    Fixes: 0fc1b09 ("tracing: Use temp buffer when filtering events")
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    rostedt committed Jan 13, 2022
  20. tracing: Fix mismatched comment in __string_len

    Here __assign_str_len() should be used for the __string_len type, instead
    of __assign_str() in the comment.
    
    Link: https://lkml.kernel.org/r/5c012db463392d0e6d4f0636203d778962ad060a.1640170494.git.geliang.tang@suse.com
    
    Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
    Fixes: 883b4ae ("tracing: Add trace_event helper macros __string_len() and __assign_str_len()")
    Signed-off-by: Geliang Tang <geliang.tang@suse.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    geliangtang authored and rostedt committed Jan 13, 2022
  21. ftrace: Add test to make sure compiled time sorts work

    Now that ftrace function pointers are sorted at compile time, add a test
    that makes sure they are sorted at run time. This test is only run if it is
    configured in.
    
    Link: https://lkml.kernel.org/r/20211206151858.4d21a24d@gandalf.local.home
    
    Cc: Yinan Liu <yinan@linux.alibaba.com>
    Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
    rostedt committed Jan 13, 2022
  22. scripts: ftrace - move the sort-processing in ftrace_init

    When the kernel starts, the initialization of ftrace takes
    up a portion of the time (approximately 6~8ms) to sort mcount
    addresses. We can save this time by moving mcount-sorting to
    compile time.
    
    Link: https://lkml.kernel.org/r/20211212113358.34208-2-yinan@linux.alibaba.com
    
    Signed-off-by: Yinan Liu <yinan@linux.alibaba.com>
    Reported-by: kernel test robot <lkp@intel.com>
    Reported-by: kernel test robot <oliver.sang@intel.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    Yinan Liu authored and rostedt committed Jan 13, 2022
  23. tracing/probes: check the return value of kstrndup() for pbuf

    kstrndup() is a memory allocation-related function, it returns NULL when
    some internal memory errors happen. It is better to check the return
    value of it so to catch the memory error in time.
    
    Link: https://lkml.kernel.org/r/tencent_4D6E270731456EB88712ED7F13883C334906@qq.com
    
    Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
    Fixes: a42e3c4 ("tracing/probe: Add immediate string parameter support")
    Signed-off-by: Xiaoke Wang <xkernel.wang@foxmail.com>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
    x2018 authored and rostedt committed Jan 13, 2022
Older