Skip to content

Releases: vladzima/volna-skillcheck

volna-skillcheck v0.3.1

Choose a tag to compare

@vladzima vladzima released this 01 Nov 19:29

Metadata and documentation update.

  • Add PyPI tags for AI agents, agent skills, LLM evaluation, prompt testing, regression testing, Codex, Claude Code, OpenCode, OpenAI-compatible APIs, Python, and Volna. Preserve the original package keywords.
  • Include the updated README, Volna origin story, recorded evaluation overview, and banner on PyPI.
  • No evaluation or adapter behavior changes.

Validation: package metadata checks, archive inspection, supported-platform CI, and the release workflow's installed-wheel smoke test.

volna-skillcheck v0.3.0

Choose a tag to compare

@vladzima vladzima released this 01 Nov 18:38

Installable agent skill and the first PyPI release.

  • Install the CLI with uv tool install volna-skillcheck, pipx install volna-skillcheck, or pip in a virtual environment.
  • Install the portable skill with npx skills add vladzima/volna-skillcheck --skill volna-skillcheck and choose your harness.
  • Ask your agent to check an instruction edit: the skill prepares representative cases, uses a bounded budget and existing provider, waits for completion, and explains the saved evidence.
  • Defaults: five cases, one trial each, up to 20 adapter calls, no automatic candidate edits or paid retries.
  • PyPI publishing uses GitHub Trusted Publishing; the build and token-bearing publish jobs are separate.

Validation: 29 offline tests, wheel installation and CLI smoke test, archive/metadata checks, workflow lint, and skill installer/reference checks for OpenCode, Codex, and Claude Code.

Limits: live skill invocation is unverified; this machine's OpenCode 2.2.5 returned an empty skill registry. The OpenCode CLI adapter was live-tested separately. Claude Code and hosted API integrations retain their existing fixture-test limits. Model judgments are advisory.

See skill installation and usage.

volna-skillcheck v0.2.0

Choose a tag to compare

@vladzima vladzima released this 01 Nov 18:19

Evaluate instruction changes through OpenCode, Claude Code, or an OpenAI-compatible Chat Completions endpoint, alongside the existing Codex adapter.

  • OpenCode V2 adapter with a private server, temporary configuration, and tools/skills/plugins/MCP denied. Live-tested with 2.2.5.
  • API adapter with configurable endpoints, environment-based keys, JSON/structured-output modes, and checks for truncated/refused responses and credential-bearing redirects. Tested against a local HTTP server, including real CLI calls; hosted providers are not live-verified.
  • Claude Code adapter using safe/restricted mode, disabled tools/hooks, and no session persistence. Fixture-tested only: Claude Code was not installed or run to verify it.
  • Shared evaluation prompts and response validation across bundled adapters.
  • A clearer README, provider setup guide, AGENTS.md/SKILL.md workflows, an optional OpenCode /skillcheck command, and a manually triggered CI template.

Python 3.10+, Linux/macOS, MIT, no Python runtime dependencies. Model judgments remain advisory; recorded-answer comparisons do not replay full agent workflows. OpenCode may retain evaluation sessions in its normal local data store.

See the README and linked adapter/workflow guides for setup. The attached wheel and source archive include the new adapters and examples; PyPI publishing is not configured.

Validation: 29 tests passed on Linux with Python 3.10, 3.12, and 3.14 and on macOS with Python 3.12. Distribution metadata, installed CLI entry points, report generation, and GitHub Markdown rendering were checked.

volna-skillcheck v0.1.0

Choose a tag to compare

@vladzima vladzima released this 01 Nov 17:57

Compare two agent instruction revisions on the same recorded tasks, then inspect blind, swapped judgments and regressions.

This first MIT-licensed release includes an installable CLI, an authenticated Codex CLI adapter, repeated trials, literal answer checks, explicit abstention, and private atomic reports that preserve partial evidence. Eight synthetic scenarios include successful baseline controls. Python 3.10+ on Linux/macOS; no Python runtime dependencies.

Download and install the attached wheel, or follow the repository README to run the examples. The source archive includes examples, tests, and documentation. PyPI publishing is not configured.

Evaluation results are advisory: this compares answers against recorded context and does not execute task tools or establish real-world agent reliability. The Codex adapter was live-tested with codex-cli 0.166.0 and uses an experimental skill-isolation flag; revalidate isolation when upgrading Codex.

Validation: 19 offline tests pass on Linux with Python 3.10, 3.12, and 3.14 and on macOS with Python 3.12. CI also builds both distributions, validates package metadata, and exercises the installed CLI outside the checkout. A separate authenticated Codex smoke evaluation completed without adapter errors.