Skip to content

v0.6.4 — production_mode: retry-aware cost

Choose a tag to compare

@justi justi released this 19 Apr 19:11
· 67 commits to main since this release
34a4697

Highlights

  • production_mode: { fallback: "..." } on compare_models / optimize_retry_policy — measures retry-aware, end-to-end cost per successful output. Each candidate runs with a runtime-injected [candidate, fallback] retry chain.
  • New metrics: escalation_rate, single_shot_cost, effective_cost, single_shot_latency_ms, effective_latency_ms, latency_percentiles — on both Report and AggregatedReport (averaged across runs:).
  • Extended ModelComparison#table: Chain / single-shot / escalation / effective cost / latency / score. Edge case candidate == fallback → em-dash (not 0%), retry injection skipped so effective == single-shot by construction.
  • context[:retry_policy_override] — new context key for transient per-call retry-policy overrides without mutating the step class.

Scope

  • Single-fallback (2-tier) chains only.
  • Step-only: raises ArgumentError if used on Pipeline::Base subclasses (pipeline-wide fallback semantics are a separate design question).

Docs