Skip to content

v0.3.0

Choose a tag to compare

@github-actions github-actions released this 15 Sep 15:11
· 244 commits to main since this release
e86ac21

Minor Changes

  • #32 ebaa5c0 Thanks @anuj-sia! - Add registered LLM-judged agent evaluations.

    Agents can register evaluation folders through evaluations/index.js. Each
    evaluation defines its judge provider and model alongside predefined metrics
    and score thresholds. Enabled evaluations score the cumulative durable session
    after every completed run and return normalized scores, reasoning, evidence,
    usage, and pass status separately from the agent result.

    Evaluation lifecycle and results are persisted as durable session events.
    Registered evaluations also run during agent tests, where judge errors and
    missed metric thresholds fail the test without changing production agent
    completion semantics.

Patch Changes

  • #32 ebaa5c0 Thanks @anuj-sia! - Run registered agent evaluations asynchronously.

    Completed agent runs now resolve execution.result without waiting for LLM
    judges. Applications can optionally await execution.evaluations, while agent
    tests continue to await all registered evaluations and enforce their metric
    thresholds. Evaluation lifecycle events and results remain durably persisted
    in the session log.