Skip to content

update eval-driven-dev skill.#1201

Merged
aaronpowell merged 2 commits intogithub:stagedfrom
yiouli:staged
Mar 29, 2026
Merged

update eval-driven-dev skill.#1201
aaronpowell merged 2 commits intogithub:stagedfrom
yiouli:staged

Conversation

@yiouli
Copy link
Copy Markdown
Contributor

@yiouli yiouli commented Mar 27, 2026

Split SKILL into multi-level to keep the skill body under 500 lines, rewrite instructions.

Pull Request Checklist

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have read and followed the Guidance for submissions involving paid services.
  • My contribution adds a new instruction, prompt, agent, skill, or workflow file in the correct directory.
  • The file follows the required naming convention.
  • The content is clearly structured and follows the example format.
  • I have tested my instructions, prompt, agent, skill, or workflow with GitHub Copilot.
  • I have run npm start and verified that README.md is up to date.
  • I am targeting the staged branch for this pull request.

Description

Update eval-driven-dev skill.

Restructured into multi-level instructions to keep the main SKILL.md under 500 lines. Rewrite various sections of the skill for better clarity.


Type of Contribution

  • New instruction file.
  • New prompt file.
  • New agent file.
  • New plugin.
  • New skill file.
  • New agentic workflow.
  • Update to existing instruction, prompt, agent, plugin, skill, or workflow.
  • Other (please specify):

Additional Notes


By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

Split SKILL into multi-level to keep the skill body under 500 lines, rewrite instructions.
Copilot AI review requested due to automatic review settings March 27, 2026 19:38
@yiouli yiouli requested a review from aaronpowell as a code owner March 27, 2026 19:38
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the eval-driven-dev skill by splitting detailed guidance into multiple reference docs and rewriting the main SKILL workflow to be shorter and more step-focused.

Changes:

  • Restructures skills/eval-driven-dev/SKILL.md into a concise 6-step workflow with pointers to new reference docs.
  • Adds new reference markdown files covering understanding the app, instrumentation, run harness patterns, dataset generation, eval tests, and investigation.
  • Updates docs/README.skills.md to reflect the new skill description and list all reference files.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
skills/eval-driven-dev/SKILL.md Rewritten main skill workflow; now references step-specific docs.
skills/eval-driven-dev/references/understanding-app.md New Step 1 deep-dive + MEMORY.md template.
skills/eval-driven-dev/references/instrumentation.md New Step 2 tactical instrumentation guidance + anti-patterns.
skills/eval-driven-dev/references/run-harness-patterns.md New Step 3 end-to-end harness patterns by app type.
skills/eval-driven-dev/references/dataset-generation.md New Step 4 dataset creation guidance + validation checklist.
skills/eval-driven-dev/references/eval-tests.md New Step 5 evaluator selection + test writing guidance.
skills/eval-driven-dev/references/investigation.md New Step 6 iteration/investigation playbook.
skills/eval-driven-dev/references/pixie-api.md Expanded API/CLI reference and evaluator catalog; adds “auto-generated” note.
docs/README.skills.md Updates the skill entry description and enumerates reference files.

Comment thread skills/eval-driven-dev/references/investigation.md
Comment thread skills/eval-driven-dev/references/pixie-api.md
Comment thread skills/eval-driven-dev/SKILL.md
Comment thread skills/eval-driven-dev/SKILL.md Outdated
Comment thread skills/eval-driven-dev/SKILL.md Outdated
Comment thread skills/eval-driven-dev/references/run-harness-patterns.md Outdated
Comment thread skills/eval-driven-dev/references/run-harness-patterns.md
Comment thread skills/eval-driven-dev/references/pixie-api.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@aaronpowell aaronpowell merged commit df0ed6a into github:staged Mar 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants