chore(autotune): improve skill with SIMD experiment lessons - #52
Conversation
## Skill improvements (from 8-iteration SIMD experiment) - Add "profile before optimizing" as explicit first step - Add "always measure end-to-end impact" rule - New Common Pitfalls section: heap alloc in hot paths, match dispatch in inner loops, variable-address loops, compiler auto-vectorization, isolated vs end-to-end mismatch - Improve subagent prompt guidance: file preservation, anti-patterns - Add profiling step to workflow reference ## Experiment log Add docs/autotune/2026-04-11-simd-tableau-msd/ with full metric.toml and log.md from the batch Clifford gates experiment (8 iterations, 3 kept, 5 discarded, 65% total improvement on MSD benchmark). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
There was a problem hiding this comment.
Pull request overview
Updates the autotune skill documentation based on lessons from a SIMD optimization experiment, and adds the full experiment ledger (metrics + findings) to docs/autotune/.
Changes:
- Add explicit “profile first” and “measure end-to-end impact” guidance to the autotune skill.
- Expand subagent prompt guidance (file preservation + include anti-patterns) and add a “Common Pitfalls” checklist.
- Add a new experiment log directory with append-only
metric.tomlandlog.mdcapturing 8 SIMD-related iterations.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| skills/autotune/SKILL.md | Enhances core rules and adds “Common Pitfalls” + improved subagent prompt guidance. |
| skills/autotune/references/autotune-workflow.md | Updates step 1 to explicitly include profiling and recording a time breakdown. |
| docs/autotune/2026-04-11-simd-tableau-msd/metric.toml | Adds the experiment’s append-only metric ledger across baseline/kept/discarded iterations. |
| docs/autotune/2026-04-11-simd-tableau-msd/log.md | Adds the detailed experiment narrative, key findings, and escalation rationale. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| [[metric]] | ||
| "commit" = "d2d5a2afbc8bf45d0db64869f563778a50a13b90" | ||
| "status" = "keep" | ||
| "description" = "Baseline measurement before SIMD optimizations" | ||
| "msd_ns" = 123886.0 | ||
| "h_ns" = 101.0 | ||
| "sqrt_x_ns" = 103.0 | ||
| "sqrt_y_ns" = 129.0 | ||
| "cz_ns" = 149.0 | ||
| "t_ns" = 156.0 | ||
| "measure_gen_ns" = 398.0 | ||
| [[metric]] | ||
| "commit" = "0b35a4e665614014c01004b8930dcb9e9c6a929c" | ||
| "status" = "discard" | ||
| "description" = "NEON MulAssign for [u64;2]: no measurable improvement, LLVM auto-vectorizes the 2-iteration loop" |
There was a problem hiding this comment.
skills/autotune/scripts/record_result.py writes a blank line between [[metric]] entries (see skills/autotune/scripts/test_autotune_scripts.py, expected output includes \n\n after each entry). This new metric.toml places entries back-to-back with no separating blank line, which diverges from the repo’s generated format and makes the ledger harder to scan / more conflict-prone. Consider reformatting to match the script output (add a blank line between each [[metric]] block).
There was a problem hiding this comment.
@copilot apply changes based on this feedback
There was a problem hiding this comment.
Added blank lines between each [[metric]] entry in metric.toml so the format matches the output produced by record_result.py (each entry ends with \n\n). Commit: 7a26efd.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…pt output format Agent-Logs-Url: https://github.com/QuEraComputing/ppvm/sessions/24901cea-3b5f-4a53-8fca-4c9d58ca741e Co-authored-by: Roger-luo <8445510+Roger-luo@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| These patterns have caused wasted iterations in past experiments. Check for them before committing to an approach. | ||
|
|
||
| - **Heap allocation in hot paths.** A `Vec::collect()` inside a per-row loop or a per-call filter can easily cost 50-100ns per allocation. Over thousands of calls, this dominates. Prefer stack-allocated fixed-size arrays (`[T; N]`) and fast-path checks that skip work entirely (e.g., skip loss-filter allocation when no qubits are lost). | ||
| - **Dynamic dispatch in inner loops.** A `match` on an enum inside a tight per-row loop destroys branch prediction. In one experiment, fusing 680 gate calls into a single loop with match dispatch was **3x slower** than 680 separate tight loops. The branch predictor handles identical branches perfectly (same gate applied 170 times) but chokes on alternating match arms. If you need fusion, use typed batch methods (one method per gate type) rather than enum dispatch. |
There was a problem hiding this comment.
This section labels a match on an enum as “dynamic dispatch”. In Rust, match dispatch is compile-time (no vtable) and the performance issue here is primarily branchy control flow / misprediction (or a jump table), not dynamic dispatch. Consider rewording the heading and text to avoid conflating these concepts (e.g., “enum/match dispatch” or “branchy dispatch in inner loops”).
| - **Dynamic dispatch in inner loops.** A `match` on an enum inside a tight per-row loop destroys branch prediction. In one experiment, fusing 680 gate calls into a single loop with match dispatch was **3x slower** than 680 separate tight loops. The branch predictor handles identical branches perfectly (same gate applied 170 times) but chokes on alternating match arms. If you need fusion, use typed batch methods (one method per gate type) rather than enum dispatch. | |
| - **Enum/match dispatch in inner loops.** A `match` on an enum inside a tight per-row loop introduces branchy control flow that can be hard for the branch predictor to learn. In one experiment, fusing 680 gate calls into a single loop with enum/match dispatch was **3x slower** than 680 separate tight loops. The branch predictor handles identical branches perfectly (same gate applied 170 times) but chokes on alternating match arms. If you need fusion, use typed batch methods (one method per gate type) rather than enum dispatch. |
Summary
Skill improvements
Experiment log
docs/autotune/2026-04-11-simd-tableau-msd/— 8 iterations, 3 kept, 5 discardedTest plan
🤖 Generated with Claude Code