cache: 7 low-severity audit findings (wave-1 batch)

[forkwright]

Migrated from forkwright/logismos-archive#70, because the archived tracker is not watched. All seven findings were re-verified against live main; crates/cache/src/flat.rs is byte-identical between the archived tree and live main, so every cited line is unchanged and every finding is still present.

Kept as one batch issue rather than seven micro-issues, per the batch's original framing — split any single row out if it gets prioritized.

Finding

Seven low-severity findings in the cache crate from the wave-1 audit, all still present: four correctness/error-category defects in FlatKvCache's byte marshalling, one endianness inconsistency, and two coverage gaps.

Evidence

correctness

• crates/cache/src/flat.rs:129 — tensor_as_bytes returns Error::ShapeMismatch when the tensor is GPU-backed, which is the wrong error category: this is a backend/location failure, not a shape failure. cache::Error has no device/backend variant at all (crates/cache/src/error.rs:9-69).

  let storage = t.cpu_storage().ok_or_else(|| Error::ShapeMismatch {
      msg: "Phase-2 FlatKvCache only accepts CPU-backed tensors".into(),
  })?;

• crates/cache/src/flat.rs:271 — len_of silently returns 0 for an out-of-range layer index, indistinguishable from a valid unwritten layer.

  self.lens.get(layer_idx).copied().unwrap_or(0)

• crates/cache/src/flat.rs:291 — cpu_storage_bytes reinterprets storage as native-endian bytes, while chunks_to_f32/f16/bf16/i32 decode them as little-endian (crates/cache/src/flat.rs:352 and siblings) — silent value corruption on a big-endian host.

• crates/cache/src/flat.rs:312 — the cpu_storage_bytes wildcard arm returns Error::ShapeMismatch for an unsupported dtype variant, again the wrong category.

  _ => Err(Error::ShapeMismatch { msg: "unsupported future CpuStorage variant".into(), }),

• crates/cache/src/flat.rs:349 — chunks_to_f32/f16/bf16/i32 use chunks_exact, silently dropping trailing remainder bytes with no postcondition check on the produced element count.

  for c in bytes.chunks_exact(4) {

testing

• crates/cache/src/flat.rs:158 — put has no test with a multi-token batch (n_tokens > 1) and value verification; every test uses a one-row tensor, including the multi-call grows_monotonically_across_layers test.
• crates/cache/src/flat.rs:226 — get has no test for a successful len = 0 read (empty read producing a zero-sized tensor). Only the error paths — a get(0,1) on an empty cache and a layer-OOB get(99,0) — are covered.

Why this matters

Individually these are low severity, but three of them compound into the same failure shape: the KV cache marshals attention state through a byte layer whose endianness contract is implicit, whose truncation is silent, and whose test coverage is single-dtype and single-token. A regression in that layer corrupts inference output with no test failure and no error.

The two mis-categorised errors are a smaller but real cost: a caller that passes a GPU-backed tensor gets told its shape is wrong, which sends debugging in the wrong direction.

Desired correction

• Add a device/backend error variant to cache::Error and use it at crates/cache/src/flat.rs:129 and crates/cache/src/flat.rs:312 instead of ShapeMismatch.
• Make len_of distinguish out-of-range from unwritten (return Option<usize> or an error).
• Settle one endianness convention across cpu_storage_bytes and the chunks_to_* readers, and document it.
• Add a postcondition check on the chunks_exact decoders that the produced element count matches the expected count.
• Add the two missing tests: a multi-token put with value verification, and a len = 0 get.

Done when: each of the seven rows above is either fixed or closed with a recorded rationale, and the endianness convention is stated once in the crate rather than implied by the readers.