Skip to content

KV-store memory + new task families

Latest

Choose a tag to compare

@samueleternity samueleternity released this 31 Oct 14:51
68770d1
Add sequence-parallel KV and delta memories (#42)

* Add sequence-parallel KV and delta memories

Add configurable causal KV and gated delta-rule memory modules for split-graph controllers, and wire them through training, checkpoint loading, and stochastic write heads. Cover causal reads, stateful execution, gradients, and backbone-compatible initialization with tests.

* Add cross-memory usage diagnostics

Extend memory dependency checks to DNC, KV, and Delta memories, including shuffled-read comparisons. Log memory usage profiles and per-cell or per-head diagnostics during evaluation.

* Add inter-layer sequence memory ablation

Add an opt-in KV/delta memory path between Mamba-1 backbone blocks, with configuration validation and inference support. Add paired graph-traversal diagnostics for memory shuffling, read lag, and layer placement on fixed ID/OOD episodes.

* Enable inter-layer memory with workspace

Allow KV/delta inter-layer memory reads alongside workspace mode. Apply the memory read after the first backbone block and its MoE residual, before the next block, and include workspace runs in memory-path probes.

* Add repeated inter-layer memory reads

Support KV/delta memory reads after every non-final Mamba-1 block, including workspace/MoE paths, with paired evaluation and checkpoint configuration. Keep workspace specialist outputs sparse by default and batch independent projections across timesteps.

* Add causal input prediction and compiled workspace

Generalize prediction objectives and error signals to non-categorical inputs, add an optional compiled workspace recurrence, and extend stochastic write heads to KV values with bounded prior statistics. Also preserve checkpoint compatibility and shorten oversized run IDs for filesystem safety.

* Add workspace capability benchmark

Introduce five synthetic task tracks with oracle-backed generation, curricula, and training/evaluation metrics. Add workspace readback bypass and shuffled-readback comparisons, plus CLI and dataset-registry integration.

* Add semantic workspace binding diagnostics

Add entity-keyed workspace writes, semantic addressing, permutation-state supervision, and readback diagnostics. Separate permutation label and sequence-length generalization evaluations, and add configurable scaling for inter-layer memory reads.

* Add transactional permutation workspace mode

Add atomic paired slot swaps and workspace-based query decoding for permutation episodes, with commit and state-accuracy metrics. Bound semantic workspace content and readback, and document the new mode.

* Canonicalize projected broadcast tensors

Make projected write values and read queries contiguous before recurrent processing so per-step batch strides remain stable for the compiler.

* Add semantic workspace writes and KV bridge

Extend semantic entity addressing to all workspace-capability task tracks with causal event-boundary writes, and add an optional two-pass workspace-to-KV side channel. Add validation, diagnostics, evaluation microbatching, documentation, and coverage for mixed-track behavior.

* Add exact Nth-farthest workspace readout

Add an optional workspace path that decodes candidate vectors from entity-addressed slots and selects the requested rank by squared distance, with entity-ID tie-breaking. Train the vector decoder with auxiliary coordinate loss and report vector and readout metrics. Document the controls and add unit and integration coverage.

* Preserve permutation diagnostics in nth-farthest mode

Include nth-farthest metadata in entity diagnostics while excluding vector targets, keeping permutation-state evaluation compatible. Add an integration test for the diagnostic path.

* Add retrieval readout and memory optimizations

Add exact workspace retrieval supervision and readout metrics, plus compact token-ID inputs and optional CPU activation offload. Reduce diagnostic overhead and semantic address-state memory, with documentation for the new options.

* linted and formatted

* test fixes

* typecheck fixes