A support desk where a document edited five minutes ago must be findable

[akash-coded]

Casebook thread. A reconstructed conversation, written for teaching. The people in it
are illustrative; every number is real and reproducible from the notebook cited below.
Replies are welcome — if you hit this yourself, add your numbers.

The constraints

• Freshness: an agent edits a KB article mid-call. It must be retrievable within 60 seconds
• Corpus: 180k articles, ~400 edits/day, bursty — a product incident causes 200 edits in an hour
• Latency: 800ms p95 end to end
• Team: two engineers, no dedicated infra

The tension

Everything about a fresh index is cheap except the embedding. A 60-second SLA on edits means
re-embedding on the write path, and the encoder is the slowest thing in the system.

Proposed design

flowchart LR
    E["article edited"] --> Q["write queue"]
    Q --> C["re-chunk, stable ids"]
    C --> H{"content hash changed?"}
    H -->|no| S["skip — no re-embed"]
    H -->|yes| B["embed, 2s batch window"]
    B --> U["upsert into live version"]
    U --> L["lexical index updated synchronously"]

Loading

Key choices:

1. Lexical updates synchronously, dense asynchronously. BM25 insert is microseconds; the
   article is findable by its own words immediately, semantically within ~10s.
2. Content-hash gate before embedding. Most "edits" are metadata or whitespace. Stable chunk
   ids make this free.
3. Upsert into the live version rather than building a new one. Freshness beats the safety of
   version aliasing here, and I am not comfortable about it.

What I am least sure about

Point 3. Every other design in this repository says build alongside, then swap the alias. A
60-second SLA does not permit that. Am I trading away the wrong thing?

[akash-coded]

Rafael · cohort peer · came from search infra

Point 3 is the right trade and you are right to be uncomfortable, but not for the reason you think.

Upserting into the live version is fine for content edits. It is not fine for encoder
changes, and the design does not distinguish them. Same operation, completely different blast
radius: a content edit touches one chunk; an encoder change invalidates every vector in the index
and there is nothing to compare against because you overwrote it.

Add one guard: the write path refuses any upsert whose embedder_tag differs from the live
version's. Encoder changes go down the build-alongside-and-swap path; content edits go straight
in. One line, and it keeps the fast path fast without letting it eat the safe path.

Without it, a partially-completed migration leaves a mixed index that returns well-formed numbers
for vectors that mean nothing to each other, and nothing errors.

[akash-coded]

Accepted, and it is the change I would not have found. The two operations look identical at the API
and have completely different recovery stories.

Moving mixed_version_check() onto the write path, not only before an alias swap.

[akash-coded]

Dana · reviewer · owns the eval harness

The design is sound. What is missing is what is missing from most freshness designs: how do you
know the SLA is being met?

"Retrievable within 60 seconds" is a claim about a distribution and nothing measures it. Before
this ships:

• A canary write. Every 5 minutes, write a document containing a unique nonce, then poll for
  it. Record time-to-findable for the lexical and dense legs separately
• p50 and p99, not a mean. The bursty case is exactly where a mean lies to you
• Alert on queue depth, not on latency. By the time time-to-findable degrades you are already
  outside the SLA; queue depth is the leading indicator

The canary is about twenty lines and it turns "we think it is fast" into a number somebody can be
woken up by.

[akash-coded]

Priya · cohort peer

Adding to the canary: reporting the two legs separately is not just diagnostics, it is the
client-facing sentence.

If lexical is 200ms and dense is 12s, the honest statement is "findable by its own words in under
a second, semantically within fifteen" — far more useful to a support manager than one number
that averages two mechanisms with different guarantees.

[akash-coded]

Both in. The two-leg sentence is what I would actually put in front of the client and I would not
have written it without this.

[akash-coded]

Marking Dana's, because a freshness design with no freshness measurement is a promise, and this
thread turned it into a system.

Critique	Verdict	Change
Write path must refuse encoder-tag mismatches	Accepted	mixed_version_check() on the write path
No measurement of the SLA	Accepted	Canary every 5 min, p50/p99, alert on queue depth
Report both legs separately	Accepted	And it becomes the client-facing sentence

Stands: synchronous lexical, asynchronous dense, content-hash gate. Nobody challenged those and
they are the load-bearing parts.

Open: the 200-edits-in-an-hour burst. The queue absorbs it, but the dense leg falls behind
exactly when those articles are most needed. Shedding? Priority by article popularity? I have no
good answer and it deserves its own thread.