Skip to content

v0.2.0-alpha.2 — Safe Learning

Pre-release
Pre-release

Choose a tag to compare

View all tags
@vikasagarwal101 vikasagarwal101 released this 27 Jun 22:08
v0.2.0-alpha.2
16536ab

bluei v0.2.0-alpha.2 — Safe Learning

The safety substrate for learned automation. Governance, evidence collection, fix validation, and statistical demotion — all shipping inert (proven by synthetic tests, unexercised on real data until alpha.3 self-seeding populates the evidence base).

⚠️ Pre-release software. Do not use in production. Use for evaluation, testing, and contributions only.

Operator Control Plane

Governance over auto-acting assets — Patterns, Recipes, AST transforms, and Emergent Rules — through a unified, event-sourced substrate:

  • Event-sourced governance trail — approval_records.jsonl is the single source of truth. Governance State (ACTIVE / PAUSED / RETIRED) is a read-time projection (most-recent-decision-wins). No separate state file. Crash-safe, auditable. No-op with empty trail.
  • Per-class policy-driven promotion gate — promote_pattern_to_recipe re-routed through an approval gate. Write-producing promotions are gate-closed by default (queue ApprovalRecord, no YAML written). In-place mutations are gate-open-with-audit. Policy file with per-asset-class per-transition mode.
  • Cascade-stage consultation — all 4 cascade stages consult Governance State. PAUSED/RETIRED assets are skipped. Per-candidate fallthrough for Patterns (PAUSED exact match falls through to structural/fuzzy).
  • Safe-mode tri-state flag — learning.mode (active / audit_only / paused). Ceilings over per-class policy. Paused freezes everything; audit_only forces gate-closed and disables SPRT firing.
  • bluei learn CLI — unified read/inbox surface: inbox (pending approvals), status (governance + native + SPRT evidence), audit (full decision history), bundle (create Golden Validation Bundle from known-good fix). Write-verbs stay native.

Golden Validation Bundles

A validation contract for learned fixes — regression fixtures, not just confidence scores:

  • Hybrid storage — product fixtures (bluei/engine/golden_bundles/) ship curated and public; repo state (repos/<name>/state/golden_bundles/) is operator-private. Privacy enforced by filesystem layout.
  • Minimal forward-compatible schema — before/after/detector-before/after/validation_command. Additive fields land in alpha.3 without breaking.
  • ruff-b904 proof fixture — first product fixture, validating the substrate end-to-end.
  • Bundle-gated promotion — Patterns with zero bundle references cannot be promoted to Recipes under gate-closed policy.

Dry Replay

Non-mutating evidence collection — record what a Pattern would have done without affecting the live worktree:

  • File-level checkpoint/restore — save original content before fix flow → restore original → apply Pattern → validate → record outcome → restore winner. Worktree state unchanged after Dry Replay.
  • Per-cycle cap — 20 attempts per cycle, prioritized oldest-first. Cap limits attempts (outcome unknown until after expensive validation).
  • dry_replay.jsonl — append-only sink with run_id, pattern_id, finding_id, would_have_outcome (HIT/MISS/FAILURE). MISS outcomes don't contribute to SPRT.

SPRT Demotion

Two-sided Wald's Sequential Probability Ratio Test replaces hardcoded demotion thresholds:

  • Risk-tolerance inputs — operators tune α (false-demotion rate), β (false-keep rate), p₀ (broken success rate), p₁ (healthy success rate). Defaults: 0.01/0.01/0.50/0.90. The "3 in 10 at 40%" emerges as a derived instance, not a magic number.
  • Structural hysteresis — LLR ≥ A → auto-promote (healthy); LLR ≤ B → auto_demote (broken). FAIL is ~2.7× more impactful than HIT.
  • Recomputed from durable stores — LLR is a pure function of dry_replay.jsonl outcomes since the most recent reset boundary. No stored accumulator; crash-safe.

Plumbing

  • run_id correlation key — UUID4 per CLI invocation, stamped on cascade_resolutions.jsonl, cost_log.jsonl, and dry_replay.jsonl. Enables cross-cycle joins.
  • Cascade-internal savings — Pattern/composite replay wins inside the cascade now contribute real $ savings (previously counts-only).

ADRs

ADR Decision
0006 All 4 asset classes governed through unified substrate
0007 Policy-driven promotion, split default (write-producing gate-closed)
0008 Governance State = projection of ApprovalRecord trail (event-sourcing)
0009 Hybrid bundle storage, filesystem-boundaried privacy
0010 run_id correlation key on ledger sinks
0011 Dry Replay scope (matched-but-not-selected, capped) + file-level checkpoint
0012 Two-sided SPRT for demotion/re-promotion; PAUSED-only effect; LLR recomputed
0013 Safe-mode tri-state flag (active/audit_only/paused)
0015 bluei learn namespace (read unified, write native)
0016 Bundle schema minimal and forward-compatible

Testing

6,245 tests (+128 from alpha.1). All synthetic — no live cycles run during alpha.2 by design.

What's Next

v0.2.0-alpha.3 "Evidence Foundation" — self-seed the substrate from vetted open-source data. The system will not run on any live repo before 0.2.0 stable, so the evidence base must be pre-populated from proven historical fixes, not auto-discovered from unvetted code.

Full changelog: CHANGELOG.md

Assets 2
Loading