Skip to content

chameleon 2.42.0

Choose a tag to compare

@github-actions github-actions released this 04 Jul 16:15

Model-tier arms in the effectiveness eval (roadmap #5). All existing lift /
no-lift evidence is sonnet-worker evidence; this makes the instrument that every
model-era decision depends on able to measure a stronger worker. Local-only eval
harness — no production hook, no user surface.

Added

  • Per-arm worker model in the effectiveness runner: --arm-model shadow=opus,enforce=fable spawns each arm's sessions on its own model (arms
    not named fall back to --model). A paired toggle arm inherits its base arm's
    model so the A/B isolates the feature, not the model. The effective model is
    recorded per cell and in run.json's new arm_models map, and
    compare_to_baseline is now model-aware: a legacy flat baseline answers only
    the sonnet arm, a model-keyed baseline is matched per arm's model, so a
    stronger-model arm never regresses against a sonnet baseline.
  • archetype_facts (CHAMELEON_ARCHETYPE_FACTS) added to the eval's env-toggle
    set, so the per-edit archetype-facts directive can be A/B'd like the other
    default-on injection features (--toggle archetype_facts).