fix(query): give retrieval a total order — results were nondeterministic

[salishforge]

Found while establishing a noise floor before the full benchmark run.

The same query returned different results

Two identical evaluations against an unchanged corpus produced different scores — Recall@3 moved 2.4pp with nothing changed.

Cause: every retrieval path ordered by relevance alone, and relevance ties constantly. Measured on the real corpus: 20 retrieved rows shared only 6 distinct ts_rank_cd values, with ties at the very top. ORDER BY rank DESC without a second key lets Postgres return tied rows in any order, and it does.

Why this matters beyond the benchmark

• The same query could return different memories on consecutive calls.
• A caller paging with LIMIT could see one row twice and miss another entirely.
• No improvement could be distinguished from noise — which would have made the upcoming full run unreproducible, and any tuning decision unfalsifiable.

Fix

id DESC as tie-break on all six ordering sites: warm-tier keyword, code, trigram and semantic, plus both shared-pool arms. Newest-first is the useful semantic — when relevance cannot separate two memories, prefer the more recent — and it's pinned by a test so a future edit can't silently invert it.

After: three consecutive evaluations returned byte-identical retrieved-id sequences, not merely equal aggregate scores.

Numbers move slightly, because the old ones were partly luck

R@3 on the stratified 42-question sample had been oscillating between 81.0% and 83.3%; it settles at 81.0%. Worth knowing that the higher figure I reported in #179 was the lucky end of that range.

Tests

tests/retrieval-determinism.test.ts — repeat-query stability, newest-first tie order, and page/limit consistency.

The fixture was interesting: it needs rows that tie on rank while differing in their first 100 characters, because query() deliberately collapses same-prefix results ("prevents similar memories filling all top-k slots"). My first attempt used identical rows and correctly failed — 25 rows deduplicated to 1. ts_rank_cd applies no length normalisation, so an unmatched leading marker leaves rank untouched; verified (6 rows → 1 distinct rank) before relying on it.

Suite	Result
retrieval-determinism	3/3
integration / http / security	24 / 43 / 29
causal-graph / abstractions	39 / 39
cache-degradation / benchmark-metrics	5 / 17

🤖 Generated with Claude Code

https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru