Skip to content

v0.2.0 — Subprocess evaluator + bundled skills

Choose a tag to compare

View all tags
@szjanikowski szjanikowski released this 22 Apr 11:45
· 86 commits to main since this release
v0.2.0
9aee34a

Highlights

  • Subprocess evaluator backends (#24). LLM-as-a-judge reviewer now runs as a subprocess against your installed claude or codex CLI instead of going through the Anthropic Agent SDK. Select with [evaluation] backend in nasde.toml. Unlocks subscription billing (Claude Max, ChatGPT Plus) for local evaluation and reuses your existing OAuth/keychain auth.
  • Bundled authoring skills + nasde install-skills. Copies the nasde-benchmark-* skills into ~/.claude/skills/ so they're available in every Claude Code session. --scope project and --force flags supported.
  • Trajectory-aware evaluation (#22). Set [evaluation] include_trajectory = true to let the reviewer access the agent's ATIF trajectory (agent/trajectory.json) alongside the final workspace — criteria can now reward how the agent got to the result.
  • MCP server configuration per variant. Drop a claude_config.json into a variant directory to wire up MCP servers for that variant's runs.
  • README rewritten for a developer audience with problem-first framing and an explicit "why NASDE" section.

Fixes

  • Evaluator captures a real stderr file descriptor, surfacing actual error output when the subprocess fails.
  • Clearer diagnostics when the evaluator's Claude Code process exits non-zero.
  • scripts/export_*_oauth_token.sh no longer use set -e, so sourcing them into a shell with an error no longer kills the terminal.
  • nasde eval now forwards the full [evaluation] config from nasde.toml (#23).

See CHANGELOG.md for full details.

Assets 2
Loading