Agent Experiment 0.9.0

Agent Experiment 0.9.0

Agent Experiment 0.9 stops re-deriving what a verdict says and records the answer instead. Agent
Judge 0.17 reads its own verdicts; this release stores that reading beside each result and deletes
the code that used to reconstruct it.

Highlights

• Pins Agent Judge to the released 0.17.0, and adopts its verdict interpretation: every item now
  records an interpretation beside its verdict, taken at record time from the live verdict.
• Removes the local re-implementation of verdict semantics — the determination walk, the copied
  requirement table, the disposition string matching, the decision and seat predicates, and the
  historical read package. 651 lines of production code removed for 191 added.
• ItemAccounting now maps a reading onto a measurement rather than deriving the reading. The
  counting rules are unchanged and remain this project's own: an abstention counts against the
  subject, a not-applicable criterion leaves the denominator, an unassessed subject is the
  instrument's failure and not the subject's, and nothing is defaulted to a pass or a failure.
• Two counting policies are now named rather than implied: a reading the record's own facts
  contradict is never counted, and a reading those facts could not confirm still is.
• Adds InterpretationReExport, which adds an interpretation to results written before the reading
  existed. It parses as a tree, compares every verdict to the one it read before writing anything,
  and is idempotent by schema version.
• Pins Agent Workflow to 0.12.3, its Agent Judge 0.17.0 compatibility release, for
  experiment-workflow.
• Retains the three published modules: experiment-core, experiment-claude, and
  experiment-workflow.

What this fixes

An item whose jury errored was recorded as a subject failure. It is an instrument failure: the
jury could not assess the subject, so the subject's outcome is unknown rather than bad. Across the
stored corpus this affected 46 stored records — 23 distinct evaluations, each written to two paths.

The same class of error is removed at its source: consumers read interpretation.reading instead of
inferring an outcome from a verdict's shape.

Compatibility

Requires Java 21. The public experiment APIs and the persisted-result projection are preserved, with
one addition: ItemResult gains a nullable interpretation.

Results written before this release remain readable and are not rewritten by it. An item whose
verdict carries no interpretation is reported as unattestable rather than being interpreted here —
an outcome derived from the verdict's shape is exactly what this release removes. Use
InterpretationReExport to add readings to an existing archive; it preserves each stored verdict
exactly, and formatting of non-ASCII may change where a file was written by an older serializer.

The normal build is credential-free. The opt-in live Claude journal integration test requires a
Claude CLI environment and may incur model cost.

License

Agent Experiment 0.9.0 is licensed under the project-specific Business Source License terms in the
repository root LICENSE file.

Maven Central

io.github.markpollack version 0.9.0: agent-experiment-parent, experiment-core, experiment-claude,
experiment-workflow. Each module carries its consumer-rooted CycloneDX SBOM (-cyclonedx.json) as a signed
attachment. The published poms declare no <repositories>; everything resolves from Maven Central.

Agent Experiment 0.8.0

Full Changelog: v0.7.2...v0.8.0

Agent Experiment 0.7.2

Agent Experiment 0.7.2

Maintenance release. Three dependencies move: Agent Judge 0.15.1 → 0.15.2, the Agent Journal
family 1.8.2 → 1.9.0, Agent Workflow 0.12.1 → 0.12.2.

This is the release the bug-fix wave was for. Agent Experiment is where the scores are produced, so
it is the component whose numbers were most affected.

If you have collected experiment scores, read this

Agent Judge 0.15.0 and 0.15.1 could silently drop a judge from a parallel jury. The jury still
returned a verdict, still named every configured judge, and did not warn — the aggregate was simply
computed from the judges that survived. It was found independently downstream, where two of six
judges turned out never to have run.

Any experiment score produced through a parallel jury on Agent Experiment 0.7.0 or 0.7.1 may have
been computed from fewer judges than it reports. Treat those scores as suspect and re-run anything
you are relying on.

The experiment reached the judge by two paths — directly, and transitively through Agent Workflow's
JudgeGate. Both are repinned here, so the resolved graph now carries one judge version throughout.
Repinning only the direct dependency would have left the defective judge arriving through workflow.

Capture columns you were missing

Agent Journal 1.9.0 adds the per-turn cost vector (input / output / thinking / cache-creation /
cache-read), vendor-neutral stop reasons, and tool-call turn ordinals and durations. Every one of
those fields was already being reported by the provider and discarded at parse time.

Capture is one-shot per run: trajectories recorded on 1.8.2 are permanently missing those columns
and cannot be backfilled. Runs recorded from 0.7.2 onward will have them.

What changed

• Repins agent-judge 0.15.1 → 0.15.2
• Repins claude-code-capture / journal-core 1.8.2 → 1.9.0
• Repins agent-workflow 0.12.1 → 0.12.2

No API change in Agent Experiment itself. All dependency changes are additive; no schema key was
renamed or removed.

Upgrading

Drop-in. Take it from AgentWorks BOM 1.19.0 or later, or bump the experiment-* coordinates to 0.7.2.

Agent Experiment 0.7.1

Agent Experiment 0.7.1

Maintenance release aligning compatible dependencies and publishing the exact certified,
consumer-rooted CycloneDX SBOM for every artifact.

Release 0.7.0

Agent Experiment 0.7.0

Agent Experiment 0.7 refreshes the Java 21 experiment framework onto the current certified
AgentWorks release line while preserving the experiment, result, and journal contracts introduced
in 0.6.

Highlights

• Updates Agent Judge to 0.15.0 and Agent Workflow to 0.12.0, including the current workflow-backed
  experiment and journal integration line.
• Updates Agent Journal/Capture to 1.8.0, Claude Code SDK to 1.5.0, and the provided-scope Agent
  Client integration to 0.29.0.
• Advances the repository-controlled Jackson 2 and Jackson 3 BOMs to 2.22.2 and 3.2.2 and refreshes
  compatible stable logging, null-safety, test, build, SBOM, and Central publishing tooling.
• Retains the three published modules: experiment-core, experiment-claude, and
  experiment-workflow.

Compatibility

This release requires Java 21. It preserves the 0.6 public experiment APIs and persisted-result
projection. Legacy Agent Experiment 0.5 / Agent Judge 0.13 result files remain readable through the
existing compatibility layer.

The normal build is credential-free. The opt-in live Claude journal integration test requires a
Claude CLI environment and may incur model cost.

License

Agent Experiment 0.7.0 is licensed under the project-specific Business Source License terms in the
repository root LICENSE file.

Release 0.6.0

Agent Experiment 0.6.0

Agent Experiment 0.6 adds workflow-backed invokers and first-class Agent Journal capture while
bringing the project onto the currently released Java 21 AgentWorks integration line.

Highlights

• Adds the published experiment-workflow module, which adapts typed Agent Workflow executions and
  journaled step cost/token metrics to the AgentInvoker contract.
• Owns the per-item Agent Journal lifecycle in AgentExperiment, including variant, item, model,
  session, tool-use, and derived step-cost evidence.
• Adds Git-backed workspace provisioning from dataset beforeRef values while preserving the
  fixture-copy provisioner.
• Records experiment-code and dataset revisions, dirty state, knowledge manifests, results,
  workspaces, and journals for auditable comparisons and re-evaluation.
• Migrates to Agent Judge 0.14's normalized Judgment and complete composite-attempt API, while
  persisting an Agent Experiment-owned result projection instead of Agent Judge implementation
  types.
• Updates to Agent Journal/Capture 1.7.0, Claude Code SDK 1.4.0, Agent Workflow 0.10.0, Agent Client
  0.26.0, Agent Judge 0.14.0, and Java 21.
• Raises the repository-governed Jackson 2, Jackson 3, and optional Logback floors to 2.21.6,
  3.1.6, and 1.5.34 respectively, closing the validated vulnerability inventory without requiring
  an unreleased Agent Journal/Capture dependency. The now-public 1.7.0 line keeps journal-core
  and claude-code-capture aligned on the same stable release and carries its corrected standalone
  Jackson closure into journal-backed experiments.
• Adds a root-only CycloneDX 1.6 aggregate SBOM attached to the parent Maven artifact with classifier
  cyclonedx, plus consistent BSL license content in packaged artifacts.
• Pins reusable build and release workflows to the approved immutable Build Tools revision and
  removes remote NVD acquisition from ordinary GitHub Actions.

Compatibility

This release requires Java 21 and moves to Agent Judge 0.14.0. ItemResult.verdict() now returns
RecordedVerdict, and JudgeExecutionDetail.candidateJudgment() returns RecordedJudgment.
Callers constructing results may still pass a live Agent Judge Verdict to the ItemResult
builder; it is converted immediately to the stable recorded projection.

Existing Agent Experiment 0.5 / Agent Judge 0.13 result files are read automatically. The migration
preserves statuses, reasoning, checks, metadata, normalized numerical meaning, categorical values,
and nested verdict evidence. It is intentionally lossy only where the old model has no normalized
equivalent: Boolean score objects collapse to their authoritative status; numerical min/max and
categorical allow-lists are not re-emitted; and unnamed subVerdicts receive synthetic legacy
attempt names because 0.13 stored no composite name, relation, or policy. Saving a loaded result
writes the new normalized format. Invalid numerical ranges and unknown legacy score shapes fail at
load time with an explicit error and must be corrected or upgraded by the producer.

The normal build is credential-free. The opt-in live Claude journal integration test requires a
Claude CLI environment and may incur model cost.

License

Agent Experiment 0.6.0 is licensed under the project-specific Business Source License terms in the
repository root LICENSE file.

Release 0.4.0

Initial release.

Release 0.3.0

Full Changelog: v0.2.0...v0.3.0

Release 0.2.0

Full Changelog: v0.1.0...v0.2.0

Release 0.1.0

Full Changelog: https://github.com/markpollack/agent-experiment/commits/v0.1.0