Add benchmark-pr skill for PR performance comparison

[chenshengxin2026]

Summary

Add a new Claude Code skill (/benchmark-pr) and two supporting slash
commands (/perf-example-device, /perf-runtime-device) that measure
the performance impact of GitHub PRs and individual examples on Ascend
hardware.

What's included

/benchmark-pr skill (.claude/skills/benchmark-pr/SKILL.md)

Given a PR number (e.g. /benchmark-pr #123 -d 4 -n 20), the skill:

1. Fetches PR metadata and computes the merge-base (fork point)
2. Pins PTO-ISA to the CI commit for reproducible builds
3. Runs tools/benchmark_rounds.sh at the merge-base (baseline)
4. Runs the same benchmark at the PR HEAD
5. Parses trimmed-mean latencies and presents a per-example comparison table
6. Flags regressions > 2% for review attention

/perf-example-device command (.claude/commands/perf-example-device.md)

Benchmarks a single example on hardware. Auto-detects an idle device via
npu-smi, runs 10 rounds, parses device logs for timing data, and reports
per-round latency with trimmed averages.

/perf-runtime-device command (.claude/commands/perf-runtime-device.md)

Benchmarks all examples under a given runtime directory on hardware.
Enumerates test cases, runs each sequentially on an idle device, and
produces a summary table with per-example average and trimmed-average
latencies.

Why

Previously there was no automated way to quantify the latency impact of a PR.
Developers had to manually checkout commits, run benchmarks, and compare
numbers by hand. These additions automate the full workflow and standardize
the reporting format.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a crucial automation tool designed to quantify the performance impact of code changes within a pull request. By establishing a new /benchmark-pr skill, developers can now automatically compare benchmark results between a PR's head and its fork point, eliminating the need for manual benchmarking and comparison. This significantly streamlines the review process for performance-sensitive changes, providing clear, standardized metrics and highlighting potential regressions.

Highlights

• New Skill Introduction: Introduced a new Claude Code skill, /benchmark-pr, to automate the performance impact assessment of pull requests.
• Automated Benchmarking Workflow: Implemented a comprehensive workflow that automatically fetches PR metadata, determines the merge-base (fork point), pins the PTO-ISA environment for reproducibility, runs benchmarks on both the merge-base (baseline) and PR HEAD, and compares the results.
• Performance Comparison and Reporting: Provided a standardized method for parsing trimmed-mean latencies, presenting a per-example comparison table, and automatically flagging performance regressions greater than 2% for review attention.
• Reproducible Environment: Ensured reproducible benchmark results by pinning the PTO-ISA environment to a specific CI commit, preventing noise from varying dependencies.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new benchmark-pr skill, which is well-documented in the SKILL.md file. The workflow is comprehensive and thoughtfully designed. My review focuses on enhancing the robustness and reliability of the shell script snippets within the documentation. I've pointed out a few areas where the implementation could be simplified and made more resilient, specifically concerning dependency management, file parsing, and handling of temporary files.