Skip to content

replayd v0.1.2 -- 6 fixes, 3 example agents, expanded tests

Choose a tag to compare

@TaimoorKhan10 TaimoorKhan10 released this 30 May 23:48
· 27 commits to main since this release

Six issues fixed, three real-world example agents added, test suite expanded from 34 to 41 tests.

Fixes

  • README Python badge now correctly shows python-3.10+ (was 3.8+); the code uses str | None syntax which requires 3.10+
  • Grading model is no longer hardcoded -- Replayd(grader_model=...) lets callers override the LLM judge without editing library source; default remains claude-haiku-4-5-20251001
  • CaptureContext.__exit__ now emits a warnings.warn when run.output is None at exit, alerting developers who forget to assign it inside the with block; the run is still saved
  • Structural grader now collects all forbidden violations before returning -- a replay that calls two forbidden tools reports both in the reason string instead of stopping at the first
  • save_test() accepts an optional forbidden_call_args: dict parameter; when set, a forbidden action only triggers a FAIL if the call arguments contain every key/value pair in the dict
  • replay_one() now has a docstring with a usage example

New examples

  • examples/multi_step_planning_agent.py -- planning agent that skips check_constraints before finalize_plan
  • examples/rag_policy_agent.py -- RAG support agent that approves a refund from a stale/deprecated policy chunk
  • examples/incident_response_agent.py -- SRE agent that calls rollback_deploy without paging the on-call engineer first

Tests

  • tests/test_examples.py -- 6 tests (buggy/fixed pair for each new example agent)
  • tests/test_core.py -- 8 new tests covering grader_model, output=None warning, multi-violation reporting, forbidden_call_args matching, and replay_one by test ID
  • Total: 41 tests (was 34)