[round · deployment-engineer shape] 45 minutes: retrieval over classified documents

[akash-coded]

Simulated interview round. A full loop in the shape a forward-deployed / deployment-engineer
panel actually runs, with the rubric attached. Written for teaching; the numbers and the
failure modes are from this repository.

The prompt, as it would be given

"Your client's analysts search a document store. Every paragraph carries a classification
marking, and markings differ within a single document. An analyst must never see a
paragraph above their clearance — and must never be able to infer its existence from the
system's behaviour. The ontology of markings changes weekly. You have four weeks and one of
their engineers. Design it."

What this shape is really testing

Not retrieval. Whether you notice that the hard constraint is not the retrieval problem.

Three things a panel of this kind is scoring:

1. Do you ask about the inference channel, or only the access channel?
2. Do you treat a weekly-changing ontology as a data-model problem or wave at it?
3. Do you say what you would do in week one versus week four, or design the finished system?

The 45 minutes, roughly

Minutes	What happens
0–5	Clarifying questions. This is scored.
5–20	Your design, out loud
20–35	Pressure on one weak point they have already chosen
35–42	"What would you do in week one?"
42–45	Your questions

The clarifying questions worth asking

• "Can an analyst know that a document exists but is above their clearance?" This is the
  whole question. If no, result counts, latency and even "no results" are inference channels.
• "Is the marking on the paragraph, or derived from the document plus the paragraph?"
• "When a marking is downgraded, does that apply retroactively to indexed content?"
• "What is the audit requirement — do you need to reconstruct who could see what, on a past date?"

The trap

Almost everyone proposes post-filtering: retrieve, then drop what the analyst may not see.

It is wrong twice. It collapses k unpredictably — ask for 10, get 3, with no signal that
happened. And it is an inference channel: the number of results is a function of content the
analyst cannot see. Ask for a rare term, get 2 results instead of 10, and you have learned that
8 classified paragraphs mention it.

Pre-filter inside the retrieval query. In this repo that is
store.lexical(..., acl_groups=...), filtering in SQL rather than after it, and it is why
k_collapse is a field in the trace.

[akash-coded]

Rafael · cohort peer · came from search infra

Adding the second-order version of the trap, because a good panel goes here and it is where I
have seen strong candidates fall over.

You pre-filter correctly, so result counts are clean. Latency is not. A query that touches a
large classified region does more index work and comes back slower, and a patient adversary can
read that. In an environment that cares about this at all, they will ask.

The answers are constant-time padding or a fixed latency floor, both of which cost real
performance — and the right move in the interview is to name the trade rather than solve it.
"Here is a channel, here is what closing it costs, this is a decision for your security team, not
for me on a whiteboard" scores better than a confident fix.

[akash-coded]

Priya · cohort peer

Also worth saying out loud that the ANN path needs separate treatment. Pre-filtering by deleting
nodes from the graph can disconnect the region containing the answer — that is
#2 — so "we pre-filter" is not one design,
it is a different design for the exact and approximate paths.

[akash-coded]

Marking this as the answer — the model response, plus the rubric.

---

A strong answer, roughly as I would give it.

"Before designing I need one thing: can an analyst know a document exists but is above their
clearance? If not, this is not an access-control problem, it is an inference-channel problem,
and it changes the design.

Assuming not. Markings go on the chunk, denormalised, and the filter goes inside the
retrieval query — never after it. Post-filtering collapses k unpredictably and leaks: result
count becomes a function of content you cannot see.

Because the ontology changes weekly, chunks carry a group key, not a marking set. The
mapping from analyst to permitted keys lives in a small, fast store resolved at query time, and
it is versioned — when audit asks in six months who could see what on 3 March, you need the
mapping as it was, not as it is.

The approximate path needs its own treatment: filtering by removing nodes can disconnect the
graph, so either filter-aware traversal or exact search inside a permitted subset, chosen on
measured recall rather than assumed.

Week one is not any of that. Week one is an eval set with clearance levels as a first-class
slice, and a permission-isolation test in CI that fails the build if a single chunk crosses a
boundary. I would not ship a retrieval change before that exists, because there would be no way
to know I had broken it."

---

How this is scored

Signal	Weak	Strong
Clarifying	Starts designing	Asks the existence-inference question first
The trap	Post-filter	Pre-filter, and names both reasons
Ontology churn	Rewrite markings on change	Group key + versioned mapping
Audit	Not mentioned	Versioned mapping, because someone will ask about a past date
Sequencing	Designs the finished system	Names what week one is, and it is the test
Honesty	Confident fix for the latency channel	Names the channel, names the cost, hands the decision back

The single highest-scoring move is putting the isolation test before the retrieval work.
Almost nobody does it, and in a regulated context it is the only part of the answer that is
non-negotiable.

Practise: EX-10
is this, in code. The design argument is
#34.