Skip to content

v1.0.0

Latest

Choose a tag to compare

@igcodinap igcodinap released this 10 Jun 14:24
· 4 commits to main since this release

Added

  • Structured agent trace model with Trace, Span, ArtifactRecord,
    StateDelta, TraceSink, WithTraceSink, and DefaultTraceSink
  • Trace linkage through Case.TraceID, Case.Trace, Result.TraceID, and
    JSONL trace_id result fields
  • Agent metrics: TaskCompletion, ToolArgumentAccuracy, PlanAdherence,
    and StepEfficiency
  • RAG metrics: ContextRecall, AnswerCorrectness, and NoiseSensitivity
  • JSON scenario definitions via LoadScenarios, DecodeScenarios, and
    BindScenarioDrivers
  • Static reports through compare.ReportHTML, compare.ReportMarkdown,
    compare.ReportJSON, and goeval report
  • Judge calibration and pairwise summaries through compare.Calibrate,
    compare.CalibrateFile, and goeval calibrate

Changed

  • Scenario steps can declare required and forbidden artifact keys directly
  • Result and trace redaction now share the same WithRedactors hooks
  • Case.TraceID now seeds empty structured trace IDs, and trace sinks write a
    shared trace ID once per Runner
  • Repeated scenario summary rows now keep all run trace IDs under
    scenario_summary.trace_ids
  • Tool-call metrics and scenario tool contracts now use trace tool-call spans
    when available, with Turns fallback for legacy cases
  • Scenario trace assembly now avoids synthesizing turn-derived tool spans when
    a step result already provides trace tool-call spans
  • Trace sink deduplication now marks a trace ID as written only after a
    successful sink write, allowing transient failures to retry
  • Judge calibration now aggregates duplicate judge or variant rows instead of
    replacing earlier rows
  • goeval report --out now rejects unknown file extensions unless --format
    is supplied explicitly
  • Markdown reports now escape table cell pipes, newlines, and HTML-sensitive
    characters