Prompt Skill Testing Workflow

[Omzig]

@bradygaster, would you mind to share with us what workflow you use to test you skills that you created?

example:

Prompt: Do the thing
Eval: Should have done the thing this way
State: success/fail

Squads response from the repo:

[Omzig]

Oh my word, i think i had an epiphany! Hire a skill tester and have it build Squad<model>.agent.md and make sure the skill tester knows to use them for testing scenarios. @bradygaster, do you use a tool or do you just let your squad figure it out?

[sytone]

I am adding that in a PR to the squad. Need to look at skills all up and the way agents are using them and how squad wants to use them

[Omzig]

yes, thank you. i have had.... success in the sense that they want to help you test, but they are not very good at checking the outcome and saying OH "prompt" results in "x" outcome, mark green. they kind of give me a soft success, then when i go try it, its less than stellar. Also they had a nifty little loop they went into and made minor changes till things kind of worked, but.