D1A 0.3.0

D1A learns from outcomes. Every decision can be logged with the model that made it; when the real outcome comes back (the tests passed, a maintainer corrected a label), D1A recalibrates on it at once and promotes a retrained model only through a statistical gate. Measured first on a coding-agent verifier: three rounds of 300 outcomes cut the calibration error from 0.068 to 0.017, and the gate kept the model each time retraining was not clearly better. This release also makes D1A more its own: Kev's playground, study harness, training-history suites and suite builders are gone (70% of the code is Kev-derived, from 77%), and Gemma 4 runs about a third faster on torch and 20–30% faster on Apple Silicon.

Added

• recipes/swe-verifier: D1A as a verifier of coding-agent runs (issue, patch and the end of the run → P(the patch resolves the issue)), on public SWE-agent runs (nebius/SWE-agent-trajectories, CC-BY-4.0) held out by repository: zero-shot and LoRA scoring, held-out evaluation with repository-bootstrap intervals and within-issue AUROC, best-of-k, a training-data filter, anytime restarts under a budget, a split-leakage study (random runs vs by issue vs by repository), the self-improvement rounds through d1a.feedback, competition-harness input (from_swegemma.py), verified references, and the paper draft.
• GET /metrics (d1a.serve): Prometheus text format: whether the model is loaded, requests and batches served, queue depth, recent batch latency (p50, p95, max), prefix-cache hits, misses and states, peak process memory and the model's device memory. Like /v1/models, it does not load an unloaded model.
• README: the self-learning loop's measured rounds (calibration error 0.068 → 0.017; the gate kept the model when retraining was not clearly better), also on the diagram.
• Self-learning as a service (d1a.serve): D1A_FEEDBACK_LOG logs every decision and returns a decision_id; POST /v1/feedback records its real outcome; D1A_OUTCOME_CALIBRATOR applies d1a.feedback's calibrator to yes/no answers and re-reads it when the file changes. /v1/models reports the learning state.
• README: Self-Learning. A section on how D1A learns from outcomes (usage, CLI, the verifier's measured numbers), step 5 in How It Works, and an animated diagram of the loop (docs/arch/learning-{light,dark}.svg from make_svgs.py).
• d1a.feedback: learning from outcomes. A FeedbackLog records each decision and, later, what actually happened (a patch passed its tests, a label was corrected); OutcomeCalibrator refits each yes/no question on those outcomes (Platt scaling on the logit, which also corrects a shifted base rate that one temperature cannot); records() turns them into training data for a small LoRA update; gate() promotes a candidate only when its held-out log loss is lower with a bootstrap interval clear of zero and no frozen suite regressed. python -m d1a.feedback status | records | calibrate. Fitted values match scikit-learn's logistic regression on D1A-E4B v0.4's verifier outputs.
• d1a.versions: the newest released checkpoint in one place. latest("JohnP1/d1a-e4b-mlx-q8") gives JohnP1/d1a-e4b-mlx-q8@v0.4; the Space takes its default from it. tests/test_versions.py fails when LATEST and the first row per model of the "Model versions" table below disagree, so a release cannot leave a default behind.
• A Hugging Face Space, ready to publish (space/, space/push.sh; not hosted yet: a Gradio or ZeroGPU Space needs HF PRO): model routing, guardrails, tool-call gating, the PR labeler (paste a GitHub PR link) and zero-shot photo and voice checks on D1A-E4B v0.4. One d1a.media.MediaModel answers all of them: MediaModel.probs now also scores a text-only record (media None) on the same weights.
• Kev's eval suites as D1A suites: external (semif-v1, typesafe-v1, wanli-v1, wanli-v2) and night2 (dates, unknowable facts, assertions) are evals/d1a/external and evals/d1a/night2, with the data byte for byte in the private dataset JohnP1/d1a-evals. d1a.benchmark --data evals/d1a/external:semif-v1 scores the same as the Kev suite did (D1A-E4B: identical probabilities on all 252 rows). python -m d1a.suites freeze --provenance records where a suite's data came from.
• D1A suites (d1a/suites.py, evals/d1a/). Our own datasets, pinned: a manifest per dataset holds its Hub commit and each partition's sha256, record count and role (train or eval), while the text stays in the private dataset. evals/d1a/<suite>:<partition> works wherever --data does (d1a.train, d1a.benchmark). The partition is fetched at the pinned commit and checked before use, and d1a.train refuses an eval partition before loading the model. pr-labels, ja-jglue and routing are frozen. The PR-labeler recipe reads them through recipes/pr-labeler/mix.py, which replaces the job's inline mixing script (byte-identical mixes, round 2's included) and records the inputs' hashes, the parameters and the mix's sha256 beside it. EXTRA now names partitions (train-ja train-blast), and DATA is gone.
• Video. POST /v1/systemone/media takes {"type": "video"}: an MP4, MOV or WebM clip up to 32 MB, of which 16 frames are sampled evenly and read through Gemma 4's own video path (timestamped frames, the same vision encoder), by the same model on MLX and in d1a.media. The sound track is not used. Zero-shot: on single-scene clips D1A-E4B v0.3 answers as it does for the still photo; questions about the order of events are not reliable yet. New dependency in the media extra: av (PyAV).
• One model for text, photos and voice. d1a.serve answers POST /v1/systemone/media (a request plus a photo or voice clip) with the model it already serves: Gemma 4's vision and audio encoders turn the media into soft tokens, and the same MLX language model reads them. The encoders (~1 GB for E4B) ship in an export's media/ folder (scripts/export_mlx.py --media, or --media-only for an existing export), and are fetched and loaded on the first photo or voice request. On the playground's 10 samples D1A-E4B v0.3 matches its own PyTorch path (0 of 20 answers change, max |Δp| 0.065). python -m d1a.media stays for PyTorch checkpoints.
• d1a.serve --idle-unload SECONDS drops the model (and the media encoders) after that long without a request and loads it again on the next one: 2.0 s for D1A-E4B on an M1 Max, which goes from 6.4 GB in use (encoders loaded) to 1.3 GB. GET /v1/models answers without loading the model and reports loaded and idle_unload_s. Default 0 (always loaded), as before.
• recipes/pr-labeler: how D1A-E4B v0.3 learned to label pull requests, as scripts: fetch labeled PRs from GitHub, build records with the labeling job's own questions, translate them into other languages (on MLX or any OpenAI-compatible server), train and score on one GPU (Hugging Face Jobs or your own), and read the results.

Changed

• Served PR requests about 20% faster on Apple Silicon (MLX). The question rows ran through all of Gemma 4's layers; its 18 KV-shared layers now run only at the positions the pointer head reads, each as a length-1 row at its own rope offset with an explicit causal (and sliding-window) mask over the shared keys (MLXDecisionModel._picked_hidden). D1A-E4B 8-bit on an M1 Max, 30 PR requests: 2,503 -> 1,955 ms per request (p50; the question rows 1,689 -> 1,013 ms). Exact in fp32 (D1A-E2B, 90 questions: max |dp| 2.3e-6); at 8 bits it moves answers less than running the rows one at a time instead of batched already does (max |dp| 0.015 vs 0.019).

• Gemma 4 trains and scores about a third faster on torch (the path HF Jobs uses). Its last 18 of 42 layers (E4B; KV-shared) take keys and values from earlier layers, so in the packed forward they now run only over the positions the pointer head reads (each question's <decide> and option ends, about 10-20 per record) instead of the whole document (d1a.backbone.Gemma4.picked_hidden). D1A-E4B v0.4 on 2 PR records (Apple M1 Max, bf16): scoring 4.84 -> 3.00 s, a LoRA training step with checkpointing 12.9-14.0 -> 8.5-8.9 s. Exact in fp32 (D1A-E2B, 120 questions: max |dp| 1.8e-6, gradients equal to 1e-5); in bf16 it moves answers less than changing the batch size does (438 questions: mean |dp| 0.0011 and 2 flips, against 0.0060 and 5 for batch 1 vs 2).

• Model families behind one interface. d1a.backbone now also decides whether the MLX backend runs a base (Gemma 4 and Qwen3.5; it was a model_type list in d1a.checkpoint) and loads Qwen3.5's CUDA serving kernels (d1a.fused_qwen35, d1a.cuda_graphs) through Qwen35.serve_cuda, so Gemma 4 is the main path and Qwen3.5 a plug-in. d1a.model.is_hybrid and d1a.model.sliding_window are gone: use d1a.backbone.for_config(config). Answers and latency are unchanged.

• PR-length requests are about 30% faster on Apple Silicon, with identical answers. The state pass (the document, run once before the questions) now stops before Gemma 4's KV-shared layers: those layers keep no cache of their own and nothing reads the state's own outputs, so for the state they were pure waste (18 of D1A-E4B's 42 layers). On an M1 Max, 60 held-out pull requests (median 1,584 state tokens): state pass 3,470 -> 2,137 ms, whole request 4,465 -> 3,082 ms (medians), every probability bit-identical; photos, voice and video too (a 16-frame clip's state pass 1,805 -> 1,079 ms). Other backbones run the full pass as before.

• Downloading a model from the Hub skips an export's media/ folder (1 GB) until a photo or voice request needs it.

Removed

• Kev's suite builders (about 6,900 lines, 93-100% Kev's): the hard-v1, breadth-v1, devtools-v1, longdoc-v1, documents-v1/v2 and transfer-v9 builders and generators, build_binding_diagnostic.py, devtools_v1_licences.json, and the tests that only exercised them. The suites stay as frozen data; tests/test_frozen_suites.py (new, in CI) checks every partition in git against its manifest's sha256 and record count, plus the eval-only suites' contracts. docs/removed-tools.md says which commit rebuilds each suite.

• Kev's training-history suites (20 of them: decision-v1, public-pool-v5/v6, round3-round15, sft-v1, sft-v2 and its rounds, transfer-v1, v3, v5, v6, v8) and six builders that only served retired or moved suites. Nothing D1A trains or evaluates on depends on them; docs/removed-tools.md says how to restore them.

• Full-weight training. d1a.train --full_ft, its FSDP2 multi-GPU mode under torchrun and the snapshot options (--snapshot_fractions, --snapshot_every_steps, --snapshot_dir) are gone, with d1a/full_ft.py, scripts/interpolate_checkpoint.py and scripts/merge_lora_checkpoint.py. Training is single-process LoRA, and LoRA training is bit-identical to before. A full-weight checkpoint is refused at load. Resume points moved to D1A's own d1a/resume.py, one file per point: a point written before this change cannot be resumed (finish that run on the older version). --shared_prefix now defaults to 0 (it was on only with --full_ft 1). training_metrics.json drops world_size and snapshots.

• Kev's unused training options. d1a.train drops --label_smoothing, --brier_w, --focal_gamma, --ord_w, --perm_kl/--perm_frac, --anchor/--anchor_w/--anchor_sources, --option_isolation and --special_embeddings: no D1A checkpoint used any of them. Training with the remaining options is bit-identical (same weights and losses on tiny Qwen3.5 and Gemma 4 bases). A checkpoint trained with option isolation is now refused at load; token-trained adapters still load. docs/removed-tools.md describes each option and how to restore it.

• Kev's in-repo playground/ (Next.js: request editor, packed vs separate, option permutation, chess) and the tools/review label-review page, about 14,000 lines, and JevPredictor in d1a/predictors.py, which ran the playground's Jev script. The demo app is jonpol01/d1a-playground; docs/removed-tools.md records what both tools did and how to restore them. The /v1/systemone/separate and /v1/systemone/permute endpoints stay.

Security

• torch 2.13.0 (from 2.8.0), which fixes three memory-corruption advisories: GHSA-vgrw-7cvw-pwgx (unpack_sequence, medium), GHSA-qfhq-4f3w-5fph (lstm_cell, low) and GHSA-rrmf-rvhw-rf47 (torch.jit.script, low). D1A calls none of these functions. On D1A-E2B's PyTorch path (Apple GPU) the answers are unchanged (0 of 40 top answers differ) and the median latency drops from 424–543 ms to 298–320 ms.

Fixed

• d1a.benchmark --data scored your own records under the training length limit (384-token states) and skipped longer ones without saying so, so a file of long documents (pull requests, reports) was scored on its short ones only. It now prints how many it skipped, and --context serving scores them all, up to what d1a.serve accepts.
• Multi-process full fine-tuning on the CPU (FSDP2 over gloo) failed on torch 2.13, which reduces gradients with PREMUL_SUM once a divide factor is set; D1A now asks for plain sums, the same result for its factor of 1.