docs: Add ADR-018 through ADR-023 and DDD for temporal tensor store#156
Merged
docs: Add ADR-018 through ADR-023 and DDD for temporal tensor store#156
Conversation
Extends ADR-017 with detailed architecture for block-based temporal
tensor compression. Introduces 6 new ADRs covering storage engine,
tiered quantization formats, temporal scoring algorithm, delta
compression/reconstruction, WASM API, and benchmarking criteria.
Adds comprehensive DDD with 5 bounded contexts mapping to the
proposed 6-crate workspace layout.
ADR-018: Block-based storage engine with BlockKey/BlockMeta model,
append-only MetaLog, tiered data files, CRC32 checksums
ADR-019: 8/7/5/3-bit quantization formats with two-level scale,
Tier0 compression-to-zero, codec_bits bit packing
ADR-020: EMA + popcount + recency scoring, hysteresis, budgeted
maintenance ticks, fast exp approximation
ADR-021: Read/write paths, sparse delta format, delta chain
compaction, factor-based reconstruction policies
ADR-022: WASM exports (tts_init/put/get/tick/stats), host-imported
BlockIO, cross-platform strategy (native/Node/browser/edge)
ADR-023: Zipf simulation acceptance test (1M blocks, 10M accesses),
microbench targets, failure mode catalog
DDD: 5 bounded contexts (Block Management, Quantization, Temporal
Scoring, Storage Engine, Delta & Reconstruction) with aggregate
roots, domain events, repository interfaces, and context map.
Total: 8,084 lines across 7 documents.
https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
…sion Implements the block-based storage engine specified in ADR-018 through ADR-023 with 5 new modules and 1 benchmark/test suite. New modules: - store.rs (1056 lines): BlockKey, BlockMeta, Tier, TieredStore with HashMap index, per-tier data storage, CRC32 checksums, eviction, and BlockIO/MetaLog/Clock traits - tiering.rs (846 lines): EMA + popcount + recency scoring with LUT-based fast_exp_neg, hysteresis, min_residency, budgeted maintenance, MigrationCandidate selection, warm aggressive mode (7->5 bit) - delta.rs (825 lines): Sparse delta format (u16 index + i16 value), DeltaChain with bounded length and compaction, FactorSet for low-rank reconstruction, encode/decode serialization - metrics.rs (770 lines): WitnessLog (ring buffer), WitnessEvent enum (Access, TierChange, Eviction, Maintenance, Compaction, etc.), StoreMetrics aggregates, StoreSnapshot serialization - store_ffi.rs (680 lines): tts_init/put/get/tick/stats/touch/evict WASM exports with u128 split into hi/lo u64, feature-gated Optimizations: - 8-bit fast path in quantizer: direct byte read/write, no bit accumulator. Dequant: 7313ns -> 1290ns (5.7x faster, 12.7 GB/s) - 8-bit fast path in bitpack: direct copy, no accumulator. Pack: 8484ns -> 742ns (11.4x), Unpack: 8845ns -> 396ns (22.3x) - #[inline] on hot functions Benchmark results (release, 16KB blocks): Quantize 8-bit: 18.9us Dequant 8-bit: 1.3us (12.7 GB/s) Quantize 3-bit: 22.5us Dequant 3-bit: 7.2us (2.3 GB/s) Score compute: 10ns Single frame decode: 178ns Segment 8-bit decode: 1.5us (11.2 GB/s) Zipf P95 read: 48ns Tier flip rate: 0.074/block/min Quality (all PASS): 8-bit: 0.39% max error 7-bit: 0.79% max error 5-bit: 3.33% max error 3-bit: 16.67% max error Tests: 170 unit + 12 integration/benchmark, all passing. https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
- Eliminate round() call in quantize hot path (1.8x speedup) - Add 3-bit dequant fast path (8-values-from-3-bytes, 2.4x speedup) - Wire WitnessLog into TieredStore (put/get/evict audit trail) - Add TieredStore.metrics() for aggregate store statistics - Add TieredStore.witness_log() accessors - Update store.get() to accept `now` tick for access tracking - 14 property-based tests (roundtrip, bitpack, segment, delta, f16, score monotonicity, extreme values, compression ratio, determinism) - 11 end-to-end integration tests (lifecycle, delta chain, quality sweep, persistence, eviction, checksum, multi-tensor, stress, compressor-to-store, factor reconstruction, witness logging) Benchmarks (4096-element tensors, release mode): 8-bit quantize: 10,745 ns (1.52 GB/s) 8-bit dequant: 992 ns (16.52 GB/s) 3-bit dequant: 2,998 ns (5.46 GB/s) Zipf P95 read: 41 ns Tier flip rate: 0.074/block/min (threshold: 0.1) All 204 tests pass. https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
- 5-bit quantize fast path (8 values → 5 bytes, no bit accumulator) - 5-bit + 7-bit dequant fast paths (8-at-a-time byte extraction) - Batch scoring: compute_scores_batch, choose_tiers_batch, score_and_partition, top_k_coldest with partial sort - SVD improvements: reconstruction_error, energy_captured, compression_ratio, from_data_adaptive (auto-rank selection) - Metrics dashboard: format_report, format_json, health_check with StoreHealthStatus, MetricsSeries with trend analysis - Core trait integration (TensorStore, TensorStoreExt, TensorStoreSnapshot) - AgentDB adapter (PatternIndex, InMemoryPatternIndex, AdaptiveTiering) - Coherence gate (CoherenceCheck, EpochTracker, verify_put) - Persistence layer (FileBlockIO, FileMetaLog, feature-gated) - Stress/fuzz tests (8 adversarial scenarios) - WASM FFI end-to-end test (feature-gated behind ffi) 306 tests pass (257 unit + 12 integration + 11 benchmarks + 14 property + 8 stress + 4 doctests). https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
…redStore - Add CoherenceCheck, EpochTracker, MetricsSeries fields to TieredStore - put() now records write epochs for staleness detection - tick() auto-records metrics snapshots for trend analysis - Add enable_coherence()/disable_coherence() + accessor methods - Add coherence_check() convenience method on TieredStore - 4 new integration tests verify wiring https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
Benchmarks (store.rs, 8 new bench tests): - Batch scoring 10k blocks vs individual scoring - 5-bit and 7-bit dequant fast paths (4096 values) - 5-bit quantize fast path (4096 values) - SVD adaptive rank selection (64x64 matrix) - format_report and format_json throughput - MetricsSeries trend computation (100 snapshots) Persistence tests (10 tests, feature-gated): - FileBlockIO: write/read, multi-tier, delete, overwrite, missing key - FileMetaLog: append/get, upsert, iter, missing key, multi-block 354 tests pass (with --features persistence). https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
ruvnet
added a commit
that referenced
this pull request
Feb 8, 2026
Added ADR links (018-023) and DDD reference for: - Block-based storage engine - Tiered quantization formats - Temporal scoring tier migration - Delta compression reconstruction - WASM API cross-platform - Benchmarking acceptance criteria Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ruvnet
added a commit
that referenced
this pull request
Feb 20, 2026
…n-adr-Lc5Do docs: Add ADR-018 through ADR-023 and DDD for temporal tensor store
ruvnet
added a commit
that referenced
this pull request
Feb 20, 2026
Added ADR links (018-023) and DDD reference for: - Block-based storage engine - Tiered quantization formats - Temporal scoring tier migration - Delta compression reconstruction - WASM API cross-platform - Benchmarking acceptance criteria Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends ADR-017 with detailed architecture for block-based temporal
tensor compression. Introduces 6 new ADRs covering storage engine,
tiered quantization formats, temporal scoring algorithm, delta
compression/reconstruction, WASM API, and benchmarking criteria.
Adds comprehensive DDD with 5 bounded contexts mapping to the
proposed 6-crate workspace layout.
ADR-018: Block-based storage engine with BlockKey/BlockMeta model,
append-only MetaLog, tiered data files, CRC32 checksums
ADR-019: 8/7/5/3-bit quantization formats with two-level scale,
Tier0 compression-to-zero, codec_bits bit packing
ADR-020: EMA + popcount + recency scoring, hysteresis, budgeted
maintenance ticks, fast exp approximation
ADR-021: Read/write paths, sparse delta format, delta chain
compaction, factor-based reconstruction policies
ADR-022: WASM exports (tts_init/put/get/tick/stats), host-imported
BlockIO, cross-platform strategy (native/Node/browser/edge)
ADR-023: Zipf simulation acceptance test (1M blocks, 10M accesses),
microbench targets, failure mode catalog
DDD: 5 bounded contexts (Block Management, Quantization, Temporal
Scoring, Storage Engine, Delta & Reconstruction) with aggregate
roots, domain events, repository interfaces, and context map.
Total: 8,084 lines across 7 documents.
https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t