Skip to content

feat(skills): add skill-creator scripts for benchmarking, optimization, and scaffolding#756

Merged
JeremyDev87 merged 1 commit intomasterfrom
feat/skill-creator-scripts-742
Mar 21, 2026
Merged

feat(skills): add skill-creator scripts for benchmarking, optimization, and scaffolding#756
JeremyDev87 merged 1 commit intomasterfrom
feat/skill-creator-scripts-742

Conversation

@JeremyDev87
Copy link
Owner

Summary

  • Add aggregate_benchmark.py: aggregates grading.json + timing.json from eval directories into benchmark.json and benchmark.md reports
  • Add run_loop.py: interactive description optimization loop with 60/40 train/test split, precision/recall/F1 metrics, and per-iteration scoring log
  • Add init_skill.sh: skill directory scaffolding with SKILL.md template (Core principle, When to Use structure), kebab-case validation, and overwrite prevention

All scripts use Python 3.8+ standard library only (no external dependencies) and support --help.

Test plan

  • aggregate_benchmark.py --help displays usage
  • aggregate_benchmark.py with mock iteration data produces valid benchmark.json (schema-conforming) and benchmark.md
  • run_loop.py --help displays usage
  • run_loop.py with missing file returns proper error
  • init_skill.sh --help displays usage
  • init_skill.sh creates SKILL.md + references/ + examples/ + scripts/ directories
  • init_skill.sh rejects invalid names and prevents overwriting existing directories
  • ajv agent schema validation passes
  • markdownlint passes

Closes #742

…n, and scaffolding (#742)

- aggregate_benchmark.py: aggregates grading.json + timing.json into benchmark.json/md
- run_loop.py: interactive description optimization loop with 60/40 train/test split
- init_skill.sh: skill directory scaffolding with SKILL.md template

Closes #742
@JeremyDev87 JeremyDev87 added feat sub-issue 상위 이슈의 하위 작업 priority:medium Medium priority skill New skill addition to .ai-rules/skills/ labels Mar 21, 2026
@vercel
Copy link

vercel bot commented Mar 21, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
codingbuddy-landing Ready Ready Preview, Comment Mar 21, 2026 1:54pm

@JeremyDev87 JeremyDev87 self-assigned this Mar 21, 2026
@JeremyDev87 JeremyDev87 merged commit e35f7ee into master Mar 21, 2026
25 checks passed
@JeremyDev87 JeremyDev87 deleted the feat/skill-creator-scripts-742 branch March 21, 2026 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat priority:medium Medium priority skill New skill addition to .ai-rules/skills/ sub-issue 상위 이슈의 하위 작업

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[D] Implement skill-creator scripts/

1 participant