PerfLens

Evidence-driven Linux performance analysis with a CLI, MCP Server, and project Skills for Codex, Claude Code, OpenCode, and local GitHub Copilot clients. 基于证据的 Linux 性能分析工具，支持 Codex、Claude Code、OpenCode 与本地 GitHub Copilot 客户端。

简体中文 | English

First installation: read Installation and first use. Do not extract the wheel; install it with pipx or uv.

After DEB installation, sudo perflens-admin setup selects the host Collector mode. perflens init then detects that safely deployed mode per project. See the Collector privilege-mode lifecycle for dry-runs, switching, rollback, and project resynchronization. See the Collector and user-space-lock roadmap for the current maturity boundary and phased extension plan. Release v0.3.1 local-Docker single-process support is specified separately in the Docker process guide. The v0.3.2 bounded build-and-optimize contract is tracked in the Docker optimization guide. The v0.4.0 Runtime Lock workflow is documented in the Runtime Lock guide. It is not part of v0.3.2 packages. A local source tag does not publish GitHub Release assets; publication requires the separate release workflow. See the trustworthy perf evidence pipeline for raw-input binding, conversion provenance, quality gates, and Agent-facing verification.

PerfLens is an evidence-driven performance-analysis toolkit for Linux applications and coding agents.

Release v0.3.2 supports Milestones 0 through 9:

• streaming FlameGraph-compatible folded stack input;
• deterministic self and inclusive hotspot aggregation;
• root-to-leaf call-path aggregation;
• symbol plus DSO grouping (DSO is explicitly unknown for standard folded input);
• bounded parse diagnostics and versioned JSON artifacts;
• a production CLI with path checks, stable error output, resource limits, and crash-durable file-and-directory-synced atomic writes;
• streaming parsing of explicitly-fielded perf script text;
• perf.data conversion through an allowlisted system perf process;
• bounded subprocess output, stderr diagnostics, timeouts, and process-group cleanup.
• ELF Build ID/debug capability inspection and verified module-offset symbolization;
• bounded workspace source context and container/build path mapping;
• generic candidate-only classification, evidence bundles, and Markdown reports.
• an official-SDK MCP server with typed, paginated tools and server-side authorization;
• a repository Performance Analysis Skill for evidence-constrained Agent workflows.
• profile and repeated-benchmark comparison with environment comparability checks;
• pyperf, Google Benchmark, and hyperfine JSON normalization;
• default-off, explicitly authorized stat/record collection. Release v0.3.0 also provides a separate target-filtered Trace Helper and verified sched/off_cpu/lock pipeline through the opt-in full_diagnostics feature profile.
• v0.3.1 project-scoped discovery and authorized collection for one process in a local Linux Docker Engine, including cgroup v2 context, bounded container symbol mapping, managed temporary test containers, and evidence-matched A/B comparison.
• v0.3.2 opt-in bounded_optimization_session: one reviewed confirmation can cover a fixed baseline build, evidence-guided collection, build snapshots whose changes are restricted to mutable_paths, up to three candidate rebuilds, and deterministic matched A/B validation under hard budgets. A non-verified final candidate is retained only after a fresh human disposition choice; that choice is recorded without upgrading the evidence verdict. The Agent/client sandbox, not PerfLens, enforces filesystem write permission.

The repository additionally contains the v0.4.0 Runtime Lock implementation: strict evidence schema 1.1 and replay verification, bounded sessions, Native pthread, Java JFR, CPython threading, and Go pprof Adapters, plus Docker optimization integration. Current local gates, reproducible packages, and the real-host matrix are recorded separately in the v0.4.0 release readiness record. Functional acceptance covers fixed JDK 17, CPython 3.13 free-threaded, Go 1.25.14/1.26.8/1.27.1 mutex/block, and Docker embedded Native evidence. Go cumulative and CPython visibility limits remain explicit. Native overhead results apply to the checked-in fixture, not arbitrary applications. Remote CI, attestation, and publication are separate from local acceptance. The source audit's Docker Artifact validation, outer-session time accounting, and MCP error reporting fixes are in candidate source and have completed rebuilt-package functional acceptance. The paging corruption-denial path remains source-regression evidence because host acceptance did not deliberately tamper with installed Artifacts. See known issues.

It does not include an AI/LLM API, Web UI, source-code patch tool, general-purpose benchmark runner, or custom agent framework. In an authorized optimization session, the external Agent edits only reviewed mutable paths and the fixed workload contract supplies correctness/Benchmark output.

PerfLens requires Python 3.12 or newer.

The commands below use a locally built v0.4.0 candidate wheel, which is not yet a published GitHub Release asset. For a published release, substitute the exact downloaded wheel filename. Install the wheel as an isolated tool:

pipx install ./perflens-0.4.0-py3-none-any.whl
# or
uv tool install ./perflens-0.4.0-py3-none-any.whl

Then opt one project in. Other projects do not see the Skill or MCP server:

cd /absolute/path/to/project
perflens init

For a project that intentionally profiles a local Docker workload, initialize the project with:

perflens init --docker

This writes a project-owned perflens-setup/container-workload.toml policy. It does not install or start Docker, join the Docker group, build/pull an image, deploy a Collector, or grant execution. The Skill uses typed MCP discovery, authorization, collection, and comparison tools only after the user confirms either one run or a bounded in-memory session.

For the v0.4.0 Runtime Lock workflow, explicitly initialize its project policy (combine both flags for Docker optimization):

perflens init --runtime-locks
perflens init --docker --runtime-locks

This enables capability discovery only. It does not instrument or attach a target, import evidence, start a workload, or access pprof. See the Runtime Lock guide.

This activates Codex and Claude Code by default. Select one of them with --client codex or --client claude-code; use --client opencode for OpenCode, or --client copilot for both Copilot CLI and VS Code Copilot Agent. Explicit --client all configures all four integrations; it does not change the no-option default. Use --read-only when the project should analyze existing evidence without automatic collection. Repeat --client to select several clients for one project. To make that selection the default for future plain init calls, run, for example, perflens client-defaults --client codex --client claude-code --client copilot. This writes strict ~/.config/perflens/config.toml; if it is absent, the built-in default remains Codex plus Claude Code. Explicit init --client ... values override it for one invocation. Claude Code and Copilot CLI share .mcp.json; onboarding validates one identical ownership copy and updates the shared entry atomically when both are selected. Rerun with perflens init --update when upgrading managed integration or changing collection gates. Update mode requires a matching setup.json, updates only recorded client entries and marked blocks, and refuses to overwrite a modified Skill or unverified client configuration. The managed perflens-setup directory is rebuilt and must not contain user files; unexpected entries cause refusal, while staged Collector assets are preserved unless regeneration is explicitly requested. With no explicit --client, update preserves the current project's recorded client set. Detach a no-longer-needed client before updating with a narrower --client selection.

Follow the generated NEXT_STEPS.md. See Installation and first use for the complete beginner flow. Onboarding safely selects the native /usr/bin or wheel /opt/perflens layout, so copy the exact generated deployment command instead of guessing paths. It installs only project-scoped integration: Codex uses .codex/config.toml, Claude Code uses .mcp.json and .claude/skills, OpenCode uses .opencode/opencode.json, and the local Copilot suite uses .mcp.json plus .vscode/mcp.json. Codex, OpenCode, and Copilot share .agents/skills. Existing unrelated configuration is preserved; user-level global configuration is not modified. Before package uninstall, preview perflens detach --project <project> --dry-run, then repeat without --dry-run. By default it removes verified selected-client MCP entries and unchanged project Skills while preserving onboarding, analysis evidence, and system Collector data. Use --keep-skills to detach MCP only or --client to select one client.

Debian 13 users can instead install the native, offline .deb packages. See Debian packages for the split main/Collector flow. Version 0.2.0 also adds explicit paranoid3_helper: an unprivileged Python Broker passes typed PID plans to a bounded root Rust Helper while perf_event_paranoid=3 remains unchanged. It is never enabled automatically and requires administrator acknowledgement of the bounded root, CAP_SYS_ADMIN, and CAP_SYS_PTRACE risk.

Release v0.3.0 adds a separate Trace Helper, in-kernel target filtering, deterministic sched/off-CPU/lock analysis, and the full_diagnostics lifecycle. The existing privileged stat/record Helper remains limited to stat/record. See the Collector and user-space-lock roadmap. Release v0.3.1 adds the local-Docker single-process target runtime. Docker support is limited to a local Linux Engine, cgroup v2, an explicit process, and fixed project policy; it excludes arbitrary Docker arguments, remote engines, Compose/Kubernetes, image build/pull, and whole-container perf aggregation. See the v0.3.1 Docker process guide. Release v0.3.2 adds the separately opt-in, benchmark-required bounded Docker optimization session. It does not build during preview, does not grant arbitrary Docker access, and never authorizes commit, push, tags, or releases. See the v0.3.2 Docker optimization guide. v0.4.0 adds bounded Runtime Lock sessions and four Adapter families, but does not support arbitrary live-process or existing-container injection. Native, CPython, and Java active collection is launch-time only; Go additionally supports an explicitly authorized same-UID literal-loopback pprof endpoint. See the Runtime Lock guide.

Run a read-only readiness summary at any time:

perflens status --project /absolute/path/to/project

Human-facing command help is Chinese-first as well. Run perflens --help, perflens setup --help, or perflens-admin --help when needed. Stable English command and option names are unchanged, and subcommand help documents duration, resource, archive-selection, and authorization boundaries.

When automatic collection is configured and local access is available, this also performs a bounded, read-only health handshake. It verifies the Collector PID/UID with kernel peer credentials and reports stale, unreachable, or wrong-identity sockets instead of declaring them ready.

Domain failures are Chinese-first for people. Automation should use the global perflens --json-errors <command> ... option or set PERFLENS_JSON_ERRORS=1 to preserve the versioned JSON error artifact. perflens doctor follows the same human-first principle: add --json for its versioned capability artifact or --output <new-file.json> to save it safely. Its five-mode output diagnoses local permission prerequisites; it does not prove that the independent Collector or any requested host mode passed real collection acceptance.

After administrator deployment and a fresh login, verify the Collector without finding a PID:

perflens accept-collector --authorize-host-acceptance

The default output is a concise Chinese pass summary with hardware-PMU, software-counting, and cpu-clock sampling status plus the evidence path, hash, metric count, and conclusion boundary. If hardware PMU evidence is not useful, automatic collection stays within the same PID, duration, and output bounds and continues with fixed software events. The result explicitly rules out IPC, hardware cache-miss, and branch-miss claims. Use --json for complete machine-readable output or --output ./collector-acceptance.json to preserve a new versioned evidence file.

With the v0.3.0 full_diagnostics profile, the same command also requires substantive target evidence, deterministic analysis, and replay verification for sched, off-CPU, and lock. Empty evidence or failed conservation does not pass. The packaged fixed eBPF filters authorized TGID/TIDs in kernel and never falls back to perf -a.

The wheel installation commands provide perflens, perflens-mcp, the optional perflens-collector, and the explicit administrator entry point perflens-admin. Confirm the release:

perflens --version
perflens-mcp --version
perflens-collector --version
perflens-admin --version

Installing directly from a source checkout is also supported:

python -m pip install .

For development with uv:

uv sync --all-groups

perflens analyze-folded \
  --input tests/fixtures/folded/normal.folded \
  --output build/analysis.json

Input follows standard folded syntax:

main;worker;parse;malloc 182
main;worker;compute 271

Frames are normalized to root → leaf. The final frame receives self weight. Every unique (symbol, DSO) in a sample receives inclusive weight once, so recursive frames cannot make a function-level inclusive percentage exceed 100%. Frame occurrences are counted separately.

Standard folded text has no DSO, PID/TID, CPU, timestamp, event, or source metadata. PerfLens records these fields as unknown; it never infers them from symbol names. Each folded line is one weighted stack record, not weight individual samples.

For existing text, generate the supported stable field set and analyze it:

perf script --ns \
  -F comm,pid,tid,cpu,time,event,period,ip,sym,dso,srcline \
  -i perf.data > profile.perf-script

perflens analyze-perf-script \
  --input profile.perf-script \
  --output build/analysis.json

Or let PerfLens run the same read-only conversion:

perflens analyze-perf-data \
  --input perf.data \
  --output build/analysis.json

analyze-perf-data never records, attaches to a process, or requests root. It invokes an absolute, allowlisted perf executable without a shell. Use --perf-path when several versions are installed and --timeout-seconds to lower the conversion deadline.

perflens inspect-elf --input build/app --output build/elf.json
perflens resolve-source \
  --binary build/app \
  --module-offset 0x1234 \
  --output build/source.json

perflens classify \
  --analysis build/analysis.json \
  --output build/diagnosis.json
perflens report \
  --analysis build/analysis.json \
  --problem "Throughput regression" \
  --metric "requests/second" \
  --output build/report.md

Source resolution requires a verified module-relative offset. A runtime IP by itself is never rebased heuristically. PerfLens prefers a long-lived llvm-symbolizer JSON provider, then falls back to a long-lived addr2line provider. Cache identity includes Build ID, module offset, and resolver version.

Classification rules label investigation candidates only. Generated reports keep direct observations, missing evidence, forbidden conclusions, and A/B validation requirements separate.

perflens compare-profiles \
  --baseline build/baseline-analysis.json \
  --candidate build/candidate-analysis.json \
  --output build/profile-comparison.json \
  --markdown-output build/profile-comparison.md

perflens normalize-benchmark \
  --input benchmark-hyperfine.json \
  --output build/benchmark.json
perflens compare-benchmarks \
  --baseline build/baseline-benchmark.json \
  --candidate build/candidate-benchmark.json \
  --output build/benchmark-comparison.json

Profile percentage changes describe the selected event distribution, not absolute elapsed time. Benchmark comparisons require repeated samples, check environment differences, apply a practical-impact threshold, and emit only candidate improvement/regression states.

Active collection is disabled by default. A CLI invocation requires both a confirmation switch and the exact per-call authorization phrase:

perflens collect-profile \
  --mode record \
  --executable /absolute/path/to/app \
  --target-arg=--workload \
  --data-output build/profile.data \
  --metadata-output build/collection.json \
  --authorize-target \
  --authorization I_EXPLICITLY_AUTHORIZE_TARGET_PROFILING

Release 0.3.0 supports record and stat; stat uses an independent typed metric adapter and derives IPC when cycles and instructions are available. The v0.3.0 full_diagnostics profile adds sched, off_cpu, and lock through a separate Trace Helper and produces dedicated verified artifacts. The existing paranoid=3 Rust Helper remains strictly stat/record-only. Package installation never enables advanced modes; an administrator selects the profile and runs real acceptance. PID attachment requires --pid, a bounded duration, --authorize-pid-attach, and the separate phrase I_EXPLICITLY_AUTHORIZE_PID_ATTACH. PerfLens never invokes sudo or changes kernel policy. See MCP server and Skill setup for the additional MCP startup gates.

For an approved live PID, PerfLens can automatically inspect permissions, create a short-lived PID-bound plan, execute it once through a separately policy-enforcing Collector Broker, and analyze the result. See automatic collection. The MCP server and Agent remain unprivileged. The Collector also enforces cumulative spool byte/file quotas and a filesystem free-space reserve; exhaustion denies new work without deleting old evidence.

Each Collector instance permits exactly one ordinary UID. Sharing its perflens group and spool across callers would expose group-readable profiles between users and is rejected.

After deployment, perflens-admin spool-status gives a read-only Chinese summary of spool usage, filesystem reserve, and currently reservable output; add --json for the versioned machine-readable artifact.

Evidence is never age-deleted automatically. Administrators can use the archive-then-prune workflow to create a bounded stored ZIP with a versioned manifest and per-file SHA-256, preserve all sources, verify both copies with a dry run, and only then explicitly authorize removal of exact matching source inodes. It selects the Broker or private Rust Helper spool from the deployed privilege mode and binds that mode and path into the manifest. The archive remains intact and Agents must not schedule pruning.

Administrators can tune the bilingual policy without memorizing a manual restart sequence: copy it to a separate mode-0600 candidate, run perflens-admin update-policy --config <candidate> --dry-run, then repeat with sudo. The command atomically applies and health-checks the policy, rolls back on activation failure, and refuses UID, fixed-spool, or privilege-mode migration. Switching between cap_perfmon and paranoid3_helper requires a reviewed undeploy and redeploy because the managed service topology is different.

Deploy and upgrade require a bounded, read-only Collector health round trip and verify the responding PID/UID through kernel credentials. A stale, wrong-owner, or unlistened socket pathname is not readiness. perflens-admin deploy prints a Chinese dry-run or success summary by default; add --json for the complete versioned artifact.

After installing a new release, run sudo perflens-admin upgrade --dry-run and then sudo perflens-admin upgrade. The explicit flow preserves policy and spool data, replaces only a verified managed unit, restarts the service, and attempts unit rollback on activation failure.

After one administrator-reviewed deployment, users do not need to discover a PID. They may ask the Skill to optimize the current project, approve one exact executable and argument list, and let the ordinary-user launcher obtain the new PID internally. The Collector still receives only a short-lived PID-bound plan; the project workload never runs with Collector privilege.

See product deployment for configurable service assets, real Collector verification, upgrades, and uninstall behavior.

The normal path is perflens init, which activates only the selected project. For separate, advanced steps, install the bundled Skill for a specific client:

perflens install-skill --project /absolute/path/to/workspace
perflens install-skill --client claude-code --project /absolute/path/to/workspace
perflens install-skill --client opencode --project /absolute/path/to/workspace
perflens install-skill --client copilot --project /absolute/path/to/workspace

The command creates .claude/skills/perflens for Claude Code or the shared .agents/skills/perflens for the other clients, and refuses to overwrite an existing Skill. To print a project-scoped MCP configuration:

perflens codex-config --workspace /absolute/path/to/workspace
perflens claude-config --workspace /absolute/path/to/workspace

Add --allow-process-execution only when perf.data conversion or source symbolization is required. Review the printed TOML before adding it to the project's .codex/config.toml.

From a source checkout, the equivalent direct registration is:

mkdir -p perflens-results
codex mcp add perflens -- \
  "$PWD/.venv/bin/perflens-mcp" \
  --allowed-root "$PWD" \
  --artifact-root "$PWD/perflens-results" \
  --allow-writes

Restart Codex, then ask:

$perflens analyze ./profile.folded and report direct evidence, candidates, and missing evidence.

For Claude Code, perflens init installs the project Skill under .claude/skills/perflens and safely merges perflens into the project .mcp.json. Claude Code asks the user to trust a project MCP server before first use. Invoke it with /perflens.

See MCP server and Skill setup for permissions, project-scoped configuration, process-execution opt-in, and the full tool flow.

Defaults are intentionally explicit:

• input file: 1 GiB;
• logical records: 10 million;
• line length: 1 MiB;
• stack depth: 4,096;
• unique frames: 2 million;
• unique call paths: 1 million;
• retained warnings: 100;
• emitted hotspots: 10,000;
• emitted call paths: 1,000.

Limits can be lowered from the CLI. Exceeding structural limits fails with a structured error rather than silently dropping exact data. Malformed individual records are skipped and reported with bounded line previews.

Prebuild the unprivileged Supervisor with the pinned Rust toolchain before the full integration suite; cold compilation must not consume pytest's per-test timeout.

Real Go/JFR integration tests also need a complete Go toolchain and JDK 17/21/25. CI stages private copies using scripts/prepare_test_runtimes.py before pytest, builds the matching cmd/pprof offline, and checks both Adapters. Locally, pass a new --directory to that script and prepend its go/bin and jdk/bin to PATH if the installed tools have unsafe permissions. The script does not change the original installations or relax production runtime checks.

cargo build --locked --package perflens-runtime-supervisor
uv run ruff check .
uv run pyright
umask 022
uv run pytest --cov=perflens
uv build
uv run pip-audit

The reproducible performance harness is:

uv run python tests/performance/benchmark_folded.py \
  --records 1000 100000 1000000 \
  --repetitions 3

See docs/performance-budget.md for the recorded environment and baseline.

See the historical v0.3.2 release readiness, the v0.4.0 release readiness record, release process, real-world profile acceptance, and known issues, and troubleshooting, the v0.3.2 Docker optimization regression playbook, and the Collector and user-space-lock roadmap, the v0.4.0 Runtime Lock workflow, the v0.3.1 Docker process roadmap, plus the perf evidence pipeline for final verification evidence, published-version workarounds, and operational failure guidance.

Chinese maintainer documentation is available in the development guide, architecture guide, compatibility matrix, known limitations, real-world acceptance record, security policy, and release-readiness record. Every document under docs/ that has an English version now links to a corresponding Simplified Chinese version.

• Folded input cannot distinguish identically named functions from different DSOs because the format omits DSO metadata.
• Percentages describe selected event weight, not wall-clock duration.
• Call paths are exact up to the configured unique-path limit.
• Symbol names are preserved with only conservative compiler-suffix cleanup.
• A hotspot is an observation, not a confirmed root cause.
• perf.data portability remains dependent on the installed perf version and access to matching DSOs/symbols; preserved unknown frames make gaps explicit.
• Active collection depends on kernel perf permissions. An ordinary process is blocked by perf_event_paranoid=3; an installed Collector may still pass its separate host acceptance, including an explicitly reported software fallback.
• Release v0.3.0 produces scheduler-delay, off-CPU interval, and low-level lock/futex-candidate artifacts through the opt-in full_diagnostics profile. It cannot promote a futex candidate to a language lock, invent owner/hold time without pairs, or cover every user-space fast path.
• Release v0.3.1 Docker collection is limited to one explicit process in a local Linux Engine with cgroup v2 and explicit project/session authorization. It excludes remote Engines, Docker Desktop VMs, Compose/Kubernetes, image build/pull, arbitrary Docker arguments, and whole-container perf aggregation; container-wide cgroup deltas are context rather than process-exclusive evidence.
• v0.4.0 Runtime Lock observes only the surface exposed by its selected Adapter and semantics. Thresholded, sampled, and cumulative inputs are not exact event logs; invisible fast paths, custom atomics, missing owners, and unpaired acquire/release events remain explicit limitations. It does not authorize arbitrary running-process or existing-container injection.