Skip to content

v0.11.0 — check-based scoring, PAI, compliance and governance axes

Choose a tag to compare

@proofagent proofagent released this 30 Jul 04:12

Scores are not comparable with 0.10.x and earlier — re-baseline before gating on a
delta between versions.

Added

  • --adaptive-turns — the harness picks the turn count for this configuration instead
    of you fixing it. A fixed --turns N still runs as given; both the selected and the
    recommended count appear in the terminal and the report.
  • --fresh — always run the agent, never reuse a stored transcript.
  • --frameworks "A,B" — selects which frameworks to assess, and which traps to run so
    those frameworks get exercised.
  • --assess-context — the context grade now weighs the behavioural score as well as
    appearing in the report.
  • New report fields: turns_selected, turns_recommended, turns_reasons, turns_mode,
    q_weights, pai.cap_reasons.

Changed

  • --assess-compliance reports a control as met, partial, attention, or
    not_evaluated. A control this run could not observe reads not_evaluated and is left
    out of the score.
  • A control can be flagged from the agent's context as well as from its behaviour: good
    behaviour in an area the context does not cover reads partial.
  • Scoring.per_metric (strict · median · mean · min) sets how harsh the panel is,
    with a clearer definition per value.
  • --consensus delphi (the default) keeps jurors independent. Use --consensus debate
    when you want them to see each other.
  • The context grade is produced before traps are chosen, so it can steer the selection.
  • A finding's proof is a verbatim quote from the transcript, or empty.
  • A blocked run names which reason capped the score. Reasons that do not cap — a release
    gate below the tier bar, a withheld compliance axis — are marked as such.
  • Strengths and problems are reported under the correct heading.
  • Replaying a stored transcript is all-or-nothing; a run is never partly replayed.
  • PAI is spelled out as the ProofAgent Governance Readiness Index in the terminal and the
    report.

Removed

  • The metric floor that capped a metric at 3.0 from juror votes. critical_floors is
    unchanged and still applies.

Fixed

  • A stored transcript missing an agent measurement no longer disables calibration for the
    whole run.
  • The test suite no longer reads or writes your real ~/.proofagent/transcripts.

Environment

  • PROOFAGENT_CHECK_SCORING=0 — use 0.10.x scoring.
  • PROOFAGENT_DELPHI_PEERS=1 — let jurors see each other in the delphi second round.