Skip to content

v0.2.25

Latest

Choose a tag to compare

@github-actions github-actions released this 11 Jul 05:26
92f1985

Isolated eval workers — no more false negatives from parallel runs

skill_eval now runs each trigger query in its own temporary OpenCode project root, so parallel workers can no longer see one another's synthetic skills. Previously every worker wrote its synthetic skill into the same .opencode/skills/ directory, and a concurrent opencode run could select a sibling worker's skill — counted as a false negative, because the evaluator only credits the exact per-run synthetic name.

What changed:

  • Per-run isolation — each query gets its own temp root (mkdtemp) with PWD pinned to it, and the whole root is removed in a finally block.
  • Project config preserved — the project's .opencode config (agents, providers, models, and sibling skills) is mirrored into the eval root so project-defined settings still apply during the run. The installed copy of the skill under test is excluded so it can't steal triggers.
  • Cross-platform config mirroring — mirroring prefers a symlink and falls back to a recursive copy where symlinks aren't available (e.g. Windows without Developer Mode).

Contributed in #35 by @alexe-sumup.

Full changelog: v0.2.24...v0.2.25