Skip to content

Validate the judge reward signal with human calibration and domain-specific evaluators #2

Description

@ctava-msft

The analysis notes that the SDK’s reward model is conceptually clear but still evidence-light: the paper specifies intent, adherence, and completion judges, but does not show that those judges actually correlate with human judgment or domain success. That is a direct risk to the learning loop because reward misspecification can cause the learner to optimize the wrong behavior.

This ticket should turn the current reward design from a plausible architecture into a validated measurement layer.

Suggested scope

Add a reward validation study that measures judge-to-human agreement.
Add support for custom evaluators so teams can plug in domain-specific checks.
Document how to evaluate whether the default judge weights are appropriate for a given workload.
Persist raw per-metric values and make it easy to reweight historical episodes without re-running judges, which the whitepaper already points to as an advantage.

Acceptance criteria

A documented calibration workflow exists for the default judges.
The SDK supports pluggable evaluators without changing the learning core.
A sample report shows how judge scores correlate with human review or task-success outcomes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions