Skip to content

New experimental memory write and read options

Choose a tag to compare

@samueleternity samueleternity released this 06 Oct 16:45
· 7 commits to main since this release
3f0e3a8
Add split-graph DNC mechanism variants (#37)

* Add split-graph DNC mechanism variants

This change adds configurable split-graph DNC read/write mechanisms: relational-mlp read scoring and a Kanerva-style closed-form write posterior, wired through controller config, CLI arguments, and model loading. It also adds graph traversal probes for relational and Q99 order-sensitivity checks, updates experiment CSV logging for Track A/B diagnostics, and preserves mechanism-aware memory resizing and link-matrix ablations.

* Relax minimum relational probe cell size

Lower the minimum required `cell_size` for relational probes from 12 to 6 so tiny smoke-test configurations can run without aborting before training. The probe only needs three role channels, and the previous 4-dimension-per-channel requirement was unnecessarily restrictive for small fixtures.

* Add relational residual DNC reader

Add a new `relational-residual` read variant that keeps the DNC cosine baseline at initialization by zero-initializing the learned residual output, and extend split-graph diagnostics with read-content entropy and max-weight metrics. Update the config and CLI/docs to expose the variant and document the Track A/B experiment logs and reader choices.

* Add configurable residual read scaling

This change introduces a configurable `--dnc-read-residual-scale` for the relational-residual DNC read variant, preserving the cosine baseline at initialization while allowing the learned correction to be strengthened or weakened. The setting is threaded through config, training, checkpoint loading, and memory resizing, and diagnostics now report the residual RMS and residual-to-cosine RMS ratio. The README is updated with the new flag and experiment guidance.

* Lower DNC residual read scale to 1.0

This updates the default relational-residual read correction from 2.0 to 1.0 across config and runtime defaults, and clarifies the docs that the residual remains cosine-anchored and exact at initialization. It also notes that lower scales such as 0.5 are available for follow-up comparisons while keeping the baseline and Track A setup consistent.

* Tune DNC residual read gating and Q99 probes

Adds a configurable cap for relational DNC residual read scores, exposed through config and CLI, and threads it through model loading and memory resizing. The Q99 order-sensitivity check is also refactored into separate fact-order and query-order probes with clearer semantic metrics and post-training gating. This keeps the residual correction bounded while making the order-sensitivity diagnostics more interpretable and robust. Includes a small .gitignore update for tmp files.

* Stabilize DNC residual normalization

Avoid zero-valued RMS denominators in residual score normalization during the first forward pass. This change adds a small epsilon before taking the square root for both residual and cosine RMS values, preventing NaNs and infinite gradients when the scorer is initialized to zero.

Note that these are purely experimental and do not replace the defaults and are not recommended to use for production yet