Skip to content

Update theme skills for preview and visual QA - #36

Merged
next-devin merged 3 commits into
mainfrom
theme-tooling-report
Jul 29, 2026
Merged

Update theme skills for preview and visual QA#36
next-devin merged 3 commits into
mainfrom
theme-tooling-report

Conversation

@next-devin

@next-devin next-devin commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

This updates next-theme-dev and next-theme-figma based on problems found during a real theme build.

The developer skill now covers standalone custom product templates, explicit product assignment, cache bypass, preview sessions, validation, screenshots, mobile layouts, and touch interactions.

The Figma skill covers empty metadata responses, large-frame slicing, asset export limits, selection sync failures, and the visual evidence required before handoff.

skills.sh status also reports version drift across Claude, Codex, and Agents installations.

Checks:

  • 80 unit tests
  • skill version checks
  • public safety checks

Related work: Theme Kit #34, Spark #39, and oscar-prime #5693.

@next-devin

next-devin commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Current rollout: Theme Kit #34, Spark #39, and Skills #36. Oscar-owned work is tracked in NextCommerceCo/oscar-prime#5693. The developer-docs draft is closed until these contracts settle.

@next-devin next-devin changed the title Theme tooling: harden agent workflows and version status Update theme skills for preview and visual QA Jul 29, 2026
@next-devin
next-devin marked this pull request as ready for review July 29, 2026 09:47
Comment thread skills.sh Outdated
status="modified"
fi
else
status="update"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: Non-strict semver versions silently fall back to status="update", contradicting the new README.

The regex gate ^[0-9]+([.][0-9]+){2}$ only matches strict X.Y.Z. Any installed or source version with a pre-release suffix (1.7.0-rc1), a missing component (1.7), or build metadata (1.7.0+build.1) skips the new comparison and lands on the old update status. The new README.md:55-58 paragraphs promise stale, modified, and local-newer rows, so users running ./skills.sh status will see plain update rows and assume nothing changed, when in reality their installed pre-release is older than the released source.

Either loosen the regex (and ignore pre-release/build in the comparison) or emit a new unknown-version status so the regression is visible.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Comment thread next-theme-dev/SKILL.md Outdated
| **Recommended** | Clone `NextCommerceCo/skills` and run `./skills.sh`; choose your local agent target and this skill. |
| **No checkout** | Use `npx skills add NextCommerceCo/skills -g --skill next-theme-dev` and add `-a <agent>` when you want a specific agent. |
| **Fallback** | Load this `SKILL.md` as a system prompt, context file, rule, or chat upload if your tool does not support native skills. |
| **Version check** | From a source checkout, run `./skills.sh status all next-theme-dev`. A `stale` row names the source and installed versions; review with `dry-run` before refreshing. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Version-check row only documents the stale row, but skills.sh now reports three distinct statuses.

README.md:55-58 documents stale, modified, and local-newer. This row only mentions stale, so an agent following the skill will not know how to interpret local-newer (newer local copy) or modified (same version, divergent files) when they appear in ./skills.sh status output. Update the row to list all three or link to the README for the full taxonomy.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Comment thread next-theme-figma/SKILL.md Outdated
| **Recommended** | Clone `NextCommerceCo/skills` and run `./skills.sh`; choose your local agent target and this skill. |
| **No checkout** | Use `npx skills add NextCommerceCo/skills -g --skill next-theme-figma` and add `-a <agent>` when you want a specific agent. |
| **Fallback** | Load this `SKILL.md` as a system prompt, context file, rule, or chat upload if your tool does not support native skills. |
| **Version check** | From a source checkout, run `./skills.sh status all next-theme-figma`. A `stale` row names source and installed versions; review with `dry-run` before refreshing. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Version-check row only documents the stale row, but skills.sh now reports three distinct statuses.

README.md:55-58 documents stale, modified, and local-newer. This row only mentions stale, so an agent running ./skills.sh status all next-theme-figma will not know how to interpret the other two statuses. Update the row to list all three or link to the README.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Comment thread next-theme-figma/SKILL.md Outdated
and rationale. DOM metrics alone are not visual QA and must never be silently
substituted for screenshots.

This runbook does not relax the per-route desktop/tablet/mobile coverage gate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Runbook insists tablet coverage is still required but its own step 5 only mandates desktop + mobile.

Step 5 (ntk capture --viewports=desktop,mobile) drops tablet, yet the closing paragraph demands every route still receive desktop/tablet/mobile coverage with explicit missing-viewport status. An agent that follows step 5 verbatim will not produce the tablet screenshots this paragraph claims are mandatory. Either drop tablet from the per-route gate or extend step 5's example to --viewports=desktop,tablet,mobile.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Comment thread next-theme-dev/SKILL.md Outdated

```bash
ntk capture --url="/?preview_theme=<theme-id>&skip_cache=1" \
--output=qa-output --viewports=desktop,mobile --json --no-progress

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: --output=qa-output is a relative path and behavior depends on the agent's CWD.

The skill does not state where the agent should run from; the destination directory will resolve against whatever cwd the agent happens to be in (frequently the theme root, repo root, or a tmp checkout). Make the example explicit (--output=./qa-output or an absolute path) or state "run from the theme root" alongside the command so QA artifacts land in a predictable location for the next handoff step.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown

Code Review Summary

Status: No Issues Found | Recommendation: Merge

All 5 issues from the previous review at cc50794f are resolved in the new commits:

  • skills.sh:160 now emits status="unknown-version" instead of silently falling back to update, and scripts/tests/test_skills_status.py adds test_status_reports_unknown_for_non_strict_versions covering 1.7.0-rc1, 1.7, and 1.7.0+build.1.
  • README.md:55-58, next-theme-dev/SKILL.md:34, and next-theme-figma/SKILL.md:33 now document stale, modified, local-newer, and unknown-version consistently.
  • next-theme-figma/SKILL.md:79 rewrites the runbook gate as desktop/mobile (with tablet recorded separately when the design supplies a tablet frame), and the matching test in next-theme-figma/tests/test_failure_mode_runbook.py:43-46 asserts the new wording.
  • next-theme-dev/SKILL.md:823-827 now says "run from the theme root" and uses --output=./qa-output so QA artifacts land predictably.

No new issues were found in the changed lines.

Files Reviewed (6 files)
  • README.md
  • next-theme-dev/SKILL.md
  • next-theme-figma/SKILL.md
  • next-theme-figma/tests/test_failure_mode_runbook.py
  • scripts/tests/test_skills_status.py
  • skills.sh
Previous Review Summary (commit cc50794)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit cc50794)

Status: 5 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 1
WARNING 3
SUGGESTION 1
Issue Details (click to expand)

CRITICAL

File Line Issue
skills.sh 160 Non-strict semver (1.7.0-rc1, 1.7, build metadata) silently falls back to status="update", contradicting the new README that promises stale/modified/local-newer rows.

WARNING

File Line Issue
next-theme-dev/SKILL.md 34 Version-check row only documents the stale row; README.md also documents modified and local-newer.
next-theme-figma/SKILL.md 33 Same documentation drift as next-theme-dev/SKILL.md:34.
next-theme-figma/SKILL.md 79 Runbook closing paragraph insists on desktop/tablet/mobile coverage but step 5 only mandates --viewports=desktop,mobile.

SUGGESTION

File Line Issue
next-theme-dev/SKILL.md 827 ntk capture --output=qa-output is a relative path; the skill does not state from which directory the agent should run.
Files Reviewed (12 files)
  • README.md - 0 issues
  • next-theme-dev/SKILL.md - 2 issues
  • next-theme-dev/tests/test_command_truth.py - 0 issues
  • next-theme-dev/tests/test_theme_tooling_report_guidance.py - 0 issues
  • next-theme-figma/SKILL.md - 2 issues
  • next-theme-figma/references/asset-export-rules.md - 0 issues
  • next-theme-figma/references/developer-workflow.md - 0 issues
  • next-theme-figma/references/figma-contract.md - 0 issues
  • next-theme-figma/tests/test_failure_mode_runbook.py - 0 issues
  • scripts/tests/test_skills_status.py - 0 issues
  • skills.json - 0 issues
  • skills.sh - 1 issue

Fix these issues in Kilo Cloud


Reviewed by minimax-m3 · Input: 23K · Output: 2.4K · Cached: 139.3K

@next-devin
next-devin merged commit 07740c5 into main Jul 29, 2026
3 checks passed
@next-devin
next-devin deleted the theme-tooling-report branch July 29, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant