Skip to content

v0.2.0

Choose a tag to compare

@SushantGautam SushantGautam released this 24 Sep 05:18
· 202 commits to main since this release
8cf9e3e

New in 0.2.0

Fine-grained execution primitives for AuditExperiment

  • run_streamed() — async generator yielding typed ExperimentEvent objects (rep_started, rep_done, model_done, cancelled)
  • run_scenario_reps() — run a single scenario N times for one model, with all streaming features
  • ExperimentEvent dataclass (exported from package root)

New constructor options

  • on_rep_done — per-rep callback (sync or async)
  • cancel_event — cooperative cancellation via asyncio.Event
  • max_retries_per_rep — auto-retry on ERROR results
  • rep_is_done — external idempotency hook (skip already-done reps)
  • max_error_reps — error threshold for cache invalidation

Improvements

  • Per-rep cache invalidation (only re-run ERROR reps, not the whole model)
  • RepeatedExperimentResults.cancelled flag (serialized in JSON)
  • Adaptive reruns emit events and call callbacks (previously silent)