You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Add Surprise-Gated Recall Phase 1 (#40)
* Add Surprise-Gated Recall Phase 1
Add a seven-lesson SGR dataset, generator validity checks, training and evaluation diagnostics, and an optional next-token prediction head using split-graph backbone features. Document Phase 0 and Phase 1 usage.
* Trace split-graph forward instability
Add targeted and on-error numeric tracing for split-graph forward boundaries, with CSV output and failed-batch replay. Prevent duplicate CfC residuals and accumulate the graph-level residual in fp32 to reduce overflow risk.
* Add console summaries for SGR diagnostics
Print run-tagged training, phase-one, and evaluation metrics alongside existing logs, including a compact episode-grouped trace of sampled token positions.
* Normalize Mamba outputs and ease lesson gates
Add non-affine output normalization to the parallel Mamba backbone, applying it after workspace deltas to preserve the standard forward path. Lower graph traversal and surprise recall lesson advancement thresholds from 85% to 80%.
* Add SGR error-gated memory and workspace paths
Add isolated Phase 2 backbone-only SGR arms and Phase 4 prediction-error consumers for DNC writes, workspace writes, and reconsideration. Include signal diagnostics, evaluation controls, and lower the surprise-recall lesson advancement threshold.
* Add split-graph backbone-only mode
Allow split-graph runs without DNC memory or its sequential combiner, while preserving backbone MoE, workspace, training, and checkpoint support. Also forward elapsed-time spans to CfC stages and keep Phase 2 Mamba residuals in fp32.
* Prevent NaNs in surprise recall paths
Run the prediction head in float32, compute loss only for valid stream positions, and mask excluded positions with `masked_fill` so NaNs cannot propagate through zero multipliers.
* Stabilize Mamba residuals and configure log intervals
Bound mixer residual RMS across Mamba paths while keeping residual streams in FP32, and apply the configured MoE residual scale to workspace specialist candidates. Add a configurable training summary interval via `--log-every`.
* Linted, formatted + a few bug fixes.
* Harden masked losses and add regression coverage
Select valid positions before cross-entropy to prevent masked non-finite logits from contaminating losses, and return differentiable zeros for empty selections. Add coverage for surprise-recall generation, next-token signals, numerical stability, and workspace diagnostics; update related documentation.
* test and linting fixes