Skip to content

pawbot v0.7.0

Choose a tag to compare

View all tags
@m2dumpling m2dumpling released this 23 Sep 13:10
· 34 commits to main since this release
v0.7.0
e1718e1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Pawbot v0.7.0 adds live Agent evaluation, optional OpenTelemetry OTLP export, and MCP SDK v2 compatibility.

Highlights

  • Run a versioned 20-case synthetic incident-triage suite with pawbot eval live list/run/compare. Live provider calls require an explicit spend cap.
  • Inspect task outcome, tool trajectory, safety, token/cost estimates, latency, and per-trial traces.
  • Enable optional OTLP spans and low-cardinality metrics; the exporter remains off by default and local Trace/replay remain available.
  • MCP uses SDK v2 discovery with a narrow legacy fallback, HTTPX2 transport, and OAuth issuer validation.
  • Trace now identifies the source used for prompt-token estimates. Tool adapters can receive operation context when they implement idempotency support.

WebUI

The existing chat, Trace, Record & Replay, and MCP management screens remain in place. This release adds no new WebUI page for live evaluation or OTLP configuration; those are operated through the CLI and configuration.

Verification

  • Final main commit e1718e1 passed the complete GitHub Actions matrix.
  • Python 3.11/Linux: 7,010 passed, 15 skipped, 82.83% coverage. Python 3.13/Windows: 6,979 passed, 46 skipped, 82.82% coverage.
  • WebUI, native TUI, wheel/package smoke, Record & Replay, Linux/macOS/Windows installer checks, and quality gate passed.
  • Wheel and sdist installed in clean Python 3.11 environments. WebUI chat, Trace, offline replay (1/1 consistent), and Microsoft Learn MCP connection smoke checks passed.
  • The manual TUI release workflow built all five platform archives and checksum files. Every archive was checked for outer and internal SHA-256, executable, third-party notices, Bun 1.3.13 license, LGPL texts, source archive, relinking instructions, and source-offer file.

Evaluation limits

The bundled cases are synthetic. The saved DeepSeek responses were regraded offline after correcting keyword false negatives, using the same responses that helped calibrate the v3 oracle. Both 60-trial reports pass against that calibrated oracle; this does not establish production task quality or an improvement in general Agent capability. The optional Judge has no independent human labels or holdout set.

Checkpoint recovery cannot undo remote side effects. Unknown write outcomes still require reconciliation or explicit approval before retry.

Assets 12
Loading