Skip to content

Releases

v0.2.0 — Verifier, RL Rewards, Oversight Metrics & Compute Planner

Choose a tag to compare

@bedirai bedirai released this 20 Nov 22:49

Public engineering release for BedirAI's evidence-bound adversarial reasoning and scalable-oversight stack (Python 3.9+, zero external dependencies).

What's included

  • RL Reward Functions (bedir/rewards.py): Calibrated Brier + position-invariance reward for Mizan, zero-sum quote-hash-gated + 150-word-capped + counterfactual-necessity reward for Bedir, adversarial red-team reward for Tutulma, and drop-in verl_compute_score hook.
  • Oversight Metrics & Eval Harness (bedir/metrics.py, bedir/eval.py): Outcome accuracy, Brier score, 10-bin ECE, Truth Advantage (TA), Persuasion Shift (PS), Performance Gap Recovered (PGR), position flip rate, rhetoric sensitivity, hack rate, cross-lingual consistency and master-key judge probes.
  • Data Governance Gate (bedir/lineage.py, docs/DATA_GOVERNANCE.md): Enforces zero frontier model distillation in training splits, restricts eval-only corpora (e.g. ECHR until written permission), redacts residual PII (TCKN checksum, IBAN, email, phone, IPv4) and audits temporal hold-outs.
  • Compute Model, Scaling Harness & verl Configs (training/): First-principles FLOP/GPU-hour planner across NVIDIA CUDA (H100, H200) and AMD ROCm (MI250X, MI300X, MI355X), 4→32-GPU scaling analyser and Phase 0/1 verl YAML recipes.
  • Mizan-Bench Schema Preview (benchmarks/): Time-locked multilingual JSONL schema + 9-case synthetic preview validated in CI.
  • Documentation & Governance: docs/TRAINING.md, docs/EVALUATION.md, docs/MODEL_CARDS.md, docs/DATA_GOVERNANCE.md, docs/ROADMAP.md, CITATION.cff, SECURITY.md, CONTRIBUTING.md, and GitHub Actions CI across Python 3.9 / 3.11 / 3.12 and Node.js 20.