v2.3.0: Autooptimize Skill & Test Harness

What's New

Autooptimize Skill (.claude/skills/autooptimize/)

Generalized version of Karpathy's autoresearch methodology for optimizing any project artifact — prompts, configs, orchestration logic, or code. Runs autonomous experiment loops: execute → eval → mutate → keep/discard.

Test Harness (autooptimize-harness/)

Comprehensive test infrastructure for measuring review-loop plugin effectiveness:

• Single-prompt testing — Test individual prompts (design-review, code-review, code-implement) via codex exec with fixtures containing known defects. ~1-5 min per run.
• End-to-end pipeline testing — Run the full review-loop via claude -p in isolated temp dirs, then execute produced code and tests automatically.
• LLM-as-judge eval — Binary pass/fail scoring using Claude as evaluator.
• 3 test scenarios — CLI calculator (simple), rate limiter (medium), KV store (complex).
• Fixtures — Design docs, code samples, and specs with known subtle defects for evaluating review quality.

Baseline Evaluation Results

Scenario	Duration	Design Rounds	Code Rounds	Tests	Status
CLI calculator	6 min	1	1	22/22	PASS
Rate limiter	95 min	6	2	32/32	PASS
KV store	~10 hr	6	2	88/88	PASS

Key findings:

• All review prompts score 100% on known-issue detection — prompt quality is already strong
• E2E pipeline produces correct, tested code across all complexity levels (142 total tests passing)
• Design stage convergence speed is the primary optimization target for complex tasks