feat(evo-testing): create skill for test creation#635
Conversation
|
There was a problem hiding this comment.
Pull request overview
Adds a new Claude “skill” document to standardize how contributors/agents generate and review component tests across the evo-web monorepo, focusing on ebayui-core (Marko) and ebayui-core-react (React) with an accessibility-first approach.
Changes:
- Introduces a new
evo-testingskill with guidance on where tests live, how to structure test plans, and how to categorize coverage (click/keyboard/focus/ARIA). - Adds example snippets for Marko tests, React tests, and Storybook interaction (
play) tests. - Documents workflow guidance to avoid duplicated tests and to align assertions with component accessibility docs.
| checkbox.focus(); | ||
| await userEvent.keyboard(" "); |
There was a problem hiding this comment.
The Marko userEvent example uses userEvent.keyboard(...), but in this repo’s Marko browser tests userEvent comes from vitest/browser and is used via a setup instance (e.g. const user = userEvent.setup(); await user.keyboard(...)). As written, the snippet doesn’t match the established pattern and is likely to be copied into tests in a non-working form.
| checkbox.focus(); | |
| await userEvent.keyboard(" "); | |
| const user = userEvent.setup(); | |
| checkbox.focus(); | |
| await user.keyboard(" "); |
| describe("when Enter key is pressed", () => { | ||
| beforeEach(async () => { | ||
| const button = screen.getByRole("button"); | ||
| button.focus(); | ||
| await user.keyboard("{Enter}"); | ||
| }); | ||
|
|
There was a problem hiding this comment.
The React keyboard unit example calls user.keyboard(...) but doesn’t define user. In this repo, keyboard input is typically done either via userEvent.keyboard(...) or by creating a setup instance (const user = userEvent.setup()) and using user.keyboard(...). Please adjust the example so it’s self-contained and copy/pasteable.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
LuLaValva
left a comment
LuLaValva
left a comment
There was a problem hiding this comment.
Thanks for adding this @reidmore-online! Since the only change is a skill, I'm comfortable merging as-is and we can tweak the specific details when there are bugs in tests created using it.
ArtBlue
left a comment
ArtBlue
left a comment
There was a problem hiding this comment.
Can we please audit this skill using skill-creator to see if there are opportunities for improvement?
Sorry, I should have mentioned I did this step :). I used skill-creator to help me take the old |
@reidmore-online , awesome! Did you want to check the evals in to establish a baseline for the skill performance or do you think it's too early for that (if this is still beta)? |
I think it's still too early, in my other experiments with evals they seem to be most helpful when you have something to compare to, and I just don't really have that yet. I am curious if I make variants of these skill that reflect the areas I have questions about, then I can possibly run evals between the different versions to see what ends up performing best. |
|
Evals are also helpful once you start making adjustments before committing potential improvements. In some cases, they are absolutely critical right off the bat. Probably not here. |
Description
Based on the previous work for the testing prompt, I turned it into a skill to build tests for evo. It currently is only setup and pointed at ebayui-core/ebayui-core-react, it will need to be updated once the other packages are ready to be included.
Notes
It is currently built to produce both unit and interactions tests for components, though right now the interactions tests cause issues, so when they are generated we aren't merging them. I am happy to keep it as is, or edit it to only do unit tests right now and develop a separate skill for the interactions tests when those are ready.
Things I'd love feedback on: