Skip to content

v1.5.0

Choose a tag to compare

View all tags
@kneelinghorse kneelinghorse released this 12 Feb 00:28
· 34 commits to main since this release
v1.5.0
3fbd6e8

Agent Vitals v1.5.0

Highlights:

  • Content-based loop detection: SHA-256 fingerprint + word-level Jaccard similarity for loop/stuck disambiguation via output_text parameter on monitor.step()
  • Framework-specific threshold profiles: Auto-detected from adapter type, with langgraph, crewai, and dspy profiles shipped in thresholds.yaml
  • ThresholdProfile dataclass and VitalsConfig.for_framework() API for per-framework threshold tuning
  • ADAPTER_FRAMEWORK_MAP for automatic profile resolution from all 8 adapters
  • New output_similarity and output_fingerprint fields on VitalsSnapshot
  • Zero regression from v1.4.0 baseline across 70-trace backtest corpus
  • CI lint fixes (removed unused imports)
  • 386 tests passing

Detection precision (cross-agent, 40 traces):

  • vitals.any: P=1.000 R=1.000
  • loop: P=0.875 R=1.000
  • stuck: P=1.000 R=0.800
  • thrash: P=1.000 R=1.000

Detection precision (combined, 70 traces):

  • vitals.any: P=1.000 R=0.982
  • loop: P=0.909 R=0.870

Content-based similarity addresses the 2 remaining loop FPs (signal-identical stuck traces) for new traces collected with output_text.

Assets 4
Loading