Debugging Tool Perfetto

Perfetto

← Debugging Tools and Commands

Perfetto combines CPU samples, scheduling, wakeups, process/thread identity, kernel events, and custom slices on one Linux timeline. Use it when separate perf, BCC, and xemu logs cannot establish event order or cross-thread dependency.

Property	Details
Supported OS	Linux tracing and browser/workstation analysis; Perfetto supports additional platforms outside this guide.
Download	Official tracebox and trace_processor launchers from get.perfetto.dev.
Verify	Run each binary with --version, then record and open a short controlled trace.
Primary outputs	.pftrace raw trace and optional symbolized .bundle.

When to use it

Perfetto is the best escalation when the unresolved question is chronological:

• Which thread ran before a slow frame?
• Where did the render thread block?
• Which thread woke it?
• How long was it runnable before receiving a CPU?
• Did GPU-driver or syscall activity overlap the dependency?
• Did one xemu event precede or follow the scheduler transition?

It does not replace perf report for the quickest on-CPU hotspot scan or BCC for a compact aggregate of blocked stacks.

Install without modifying SteamOS

The official Linux cookbook supplies self-contained download launchers. Put them in a user-owned tools directory rather than altering the SteamOS root filesystem:

mkdir -p "$HOME/Tools/perfetto"
cd "$HOME/Tools/perfetto"

curl -LO https://get.perfetto.dev/tracebox
curl -LO https://get.perfetto.dev/trace_processor
chmod +x tracebox trace_processor

./tracebox --version
./trace_processor --version

Record the resolved download URL, version, and SHA-256. Review downloaded launchers before using them in an automated appliance. A later SteamOS/kernel update requires another short admission check.

CPU, scheduler, and wakeup capture

Save this as xemu-cpu.cfg:

duration_ms: 10000

buffers {
  size_kb: 65536
  fill_policy: DISCARD
}

data_sources {
  config {
    name: "linux.perf"
    perf_event_config {
      timebase {
        counter: SW_CPU_CLOCK
        frequency: 199
        timestamp_clock: PERF_CLOCK_MONOTONIC
      }
      callstack_sampling {
        scope {
          target_cmdline: "xemu"
        }
        kernel_frames: true
      }
    }
  }
}

data_sources {
  config {
    name: "linux.ftrace"
    ftrace_config {
      ftrace_events: "sched/sched_switch"
      ftrace_events: "sched/sched_waking"
    }
  }
}

data_sources {
  config {
    name: "linux.process_stats"
    process_stats_config {
      scan_all_processes_on_start: true
    }
  }
}

Capture:

sudo "$HOME/Tools/perfetto/tracebox" \
  -c xemu-cpu.cfg \
  --txt \
  -o xemu-thread.pftrace

The configuration is intentionally bounded to ten seconds. Reduce frequency or duration if overhead changes the symptom. Increase the buffer only after observing drops; do not silently turn every trace into an unbounded system capture.

Symbols

Native call stacks need the exact optimized xemu binary and matching debug information. Bundle userspace symbols on a machine with llvm-symbolizer and access to the captured binary/debug paths:

"$HOME/Tools/perfetto/trace_processor" bundle \
  --symbol-paths /path/to/xemu/build,/path/to/debug/files \
  xemu-thread.pftrace \
  xemu-thread.bundle

Open the bundle in the Perfetto UI. Keep the raw trace even when bundling succeeds. A symbolized name without matching build identity is not valid evidence.

Scheduling-event call stacks

Periodic sampling can miss the exact stack where a thread blocked. Perfetto's Linux cookbook supports call-stack sampling triggered by sched_switch and sched_waking. Use that mode when the scheduler timeline identifies the interval but periodic stacks do not identify the blocking/waking paths.

Keep this as a second, targeted capture. Do not combine every ftrace source, syscall, and high-rate sample source in the first run.

Reading the timeline

For one suspected frame or interval:

1. select the exact xemu process;
2. find the target thread and confirm its run-local TID/name;
3. separate running slices, blocked intervals, and runnable delay;
4. follow sched_waking to the waker;
5. inspect stacks at switch-out and wakeup boundaries;
6. correlate with xemu's guest-progress anchor;
7. inspect driver/GPU activity only as correlation until a dependency is demonstrated.

Perfetto makes simultaneous events visible. It does not prove that one event caused another merely because they overlap.

Optional xemu instrumentation

Custom Perfetto/TrackEvent slices could eventually mark guest flips, PFIFO service, render submission, surface synchronization, and audio frames. Such instrumentation requires a separate reviewed implementation and an explicit disabled-path cost check.

Do not add permanent high-frequency events solely to make a one-off trace easier. First use scheduler/process data already supplied by the kernel.

Steam Deck notes

The checked Deck did not have tracebox installed, but it had tracefs, debugfs, BTF, and the required scheduler tracepoints exposed to root. The user-owned launcher approach avoids changing the read-only SteamOS system image. Scheduler and kernel call-stack capture require sudo on that appliance.

The Deck's kernel.perf_event_paranoid value was 2. The official Perfetto examples lower it for some system-wide/call-stack configurations. Prefer running the bounded capture with elevation first; do not permanently weaken system policy just to simplify commands. If a temporary sysctl change is unavoidable, record the original value and restore it immediately.

Boundaries and common mistakes

• Perfetto tracing is diagnostic overhead, not an acceptance benchmark.
• A browser UI view is not enough evidence; retain the raw trace/config and exact selected interval.
• Full-system traces can contain private process names, paths, command lines, and application activity.
• A target command line must match the actual executable name.
• Missing symbols can make correct stacks appear unrelated.
• sched_switch/sched_waking show scheduling relationships; they do not replace guest-state inspection.

Evidence to retain

• tracebox/trace_processor version and SHA-256;
• kernel/SteamOS version and privilege/sysctl state;
• exact text config and command;
• raw .pftrace, optional bundle, and lost-event warnings;
• xemu executable/symbol hashes and capture interval;
• a small exported screenshot or query only as an index into the retained raw evidence.

Official references

• Perfetto Linux tracing cookbook
• Perfetto CPU scheduling data source
• Perfetto tracebox reference
• Perfetto symbolization
• Perfetto UI

Related xemu guides

• Linux perf
• BCC and eBPF
• Steam Deck and Linux Thread Diagnosis
• xemu cause counters