v0.3.1 — NNC graph storage and Tensor kernel efficiency

v0.3.1 — NNC graph storage and Tensor kernel efficiency

v0.3.1 (2026-10-28) optimizes NNC compilation and the matched
operator kernels to keep Tensor operands in SRAM and reduce scalar command
overhead. The supported quantization formats introduced in v0.3
remain available. E3, Edge-32 and ASIC RTL revisions are unchanged.

Main changes

• Compiler-owned storage planning. NNC propagates tensor-engine storage
  across compatible graph edges, inserts DMA layout boundaries when needed,
  and expands Attention into Matmul-transpose, softmax and Matmul. Logical
  PyTorch shapes remain unchanged; io_layout.json describes physical I/O.
• Runtime Matmul DMA slicing. BF16 RHS stays row-major in DRAM and can
  change between invocations. Hardware strided circular DMA slices 8×8 tiles
  into a fixed eight-slot, 1 KiB SRAM ring. Token chunks bound SRAM usage.
• Less kernel control overhead. Matmul reuses PSUM/intermediate-output
  pointers; Matmul-transpose reuses input pointers across output blocks.
  Separate K-boundary loops keep tile modes constant, and metadata passes by
  const reference. Dimensions remain runtime values. Arithmetic, DMA order,
  synchronization and timing windows are unchanged by this control change.
• Unified quant Linear calls. Six named quantized Linear APIs use NNC
  byte payloads and the shared slot kernel. Generated calls use planned
  tensor storage; quant_token_capacity=128 selects the benchmark ceiling.
• Explicit cache ownership. ASIC/DMA kernels synchronize their engines;
  scalar producers publish payload writes and scalar readers acquire DMA
  results at ownership boundaries. See the
  memory contract.

Operator MAC utilization

The NNC versioned benchmark report
contains all 24 rows, source hashes and measurement provenance. The public
edge-cores simulator reproduced every row exactly on 2026-10-28, with exact
BF16 output comparison. The release validation record
records the consuming workspace and commands.

Workload: 64×64 weights/RHS, 128 tokens, BF16 input/output, no warmup.
Ideal work is 524,288 MACs / 64 MACs per cycle = 8,192 cycles. At token
capacity 128, each operator uses one chunk:

Operator	Stream cycles	Stream MAC utilization	Operator cycles	Operator MAC utilization
Matmul	8,725	93.89%	9,329	87.81%
Matmul-transpose	8,725	93.89%	9,243	88.63%
NVFP4 Linear	8,833	92.74%	9,423	86.94%
MXFP4 Linear	8,861	92.45%	9,403	87.12%
Q8_0 Linear	8,765	93.46%	9,210	88.95%
FP4 Linear	8,747	93.65%	9,186	89.18%
FP8 E4M3FN Linear	8,765	93.46%	9,210	88.95%
FP8 E5M2 Linear	8,765	93.46%	9,210	88.95%

Utilization is ideal cycles divided by measured cycles. Stream timing starts
before weight/RHS DMA submission and ends after final Tensor/DMA sync.
Operator timing also includes kernel validation and setup. Ordinary model
input/output boundary transfers and KV-cache traffic are excluded. These
are RTL simulation measurements without a clock-frequency assumption.

The report's controlled A/B comparison isolates the final Matmul control
optimization: NNC 7f0946a → 79091dc, with the same RTL and workload.
At capacity 32, Matmul operator utilization improves 50.05% → 66.07%
(16,367 → 12,399 cycles), and Matmul-transpose 44.62% → 57.31%
(18,358 → 14,293 cycles). At capacity 128, the corresponding improvements
are 86.75% → 87.81% and 86.85% → 88.63%. This is a comparison within
the NNC optimization series, not a measured v0.3-to-v0.3.1 speedup for every
operator or a complete-model speedup claim.

Compatibility and integration

Regenerate model artifacts with the pinned compiler/runtime pair. External
payloads may now use [channel/8, token, 8] order, with an outer head axis
where applicable; consumers must follow io_layout.json. Matmul RHS remains
row-major BF16 in DRAM. The six saved Linear examples include regenerated
headers, payloads, goldens and layout metadata. The pre-update saved examples
were also replayed successfully against the new runtime before regeneration.
The flat Llama example's tracked headers and layout metadata were also
regenerated and its public runner passed output comparison and profiling.

CPU-produced payloads must be published before DMA; CPU output readers must
acquire completed results. See Matmul and
quant Linear for shape, SRAM and
numerical limits.

Release integration restores the MXFP4/NVFP4 example sources and smoke
entries, preserves editable NNC installation, repairs the NNC lockfile's
extra markers, links shared freestanding metadata-copy support in workspace
examples, and adapts the direct Matmul sentinel to the new storage contract.
The public simulator supports opt-in +profile_mac and +trace_cmd outputs,
so the upstream benchmark can check actual MAC issues, accepted STARTs and
DMA descriptors without private test repositories. Cache ownership coverage
is included in the existing NNC regression adapter.

Validation and source versions

See v0.3.1 validation for the final full regression,
benchmark and setup results, exact component revisions and local evidence.
The release pins NNC 645cc3793c1b7a309314a5e5adb3bdce9cf7556a, including
the compatibility fixes above. All other submodule revisions are recorded
in the validation document. Clone recursively, or run
git submodule update --init --recursive after checking out v0.3.1,
to obtain the pinned dependencies; GitHub source archives omit submodules.