Can I use these numbers in a client conversation?

[akash-coded]

Casebook thread. A reconstructed conversation, written for teaching. The people in it
are illustrative; every number is real and reproducible from the notebook cited below.
Replies are welcome — if you hit this yourself, add your numbers.

The question in one line

Can I quote the cost-per-query and recall figures from these notebooks to a client?

Where

Notebooks 06 and 07

[akash-coded]

The numbers, no. The arithmetic and the method, absolutely — and that is the more valuable
half anyway.

Three reasons the absolute values do not transfer:

1 · The corpus is synthetic. Generated from a fact graph so gold evidence is true by
construction, which is excellent for teaching measurement and means nothing about your client's
documents. It is also balanced by construction, which is why one of the deck's failure modes
does not reproduce here.

2 · The components are stand-ins. LSA rather than a modern encoder; a logistic regression
rather than a trained cross-encoder; an extractive reader rather than a model. Notebook 00
prints the full inventory. Recall figures are not comparable to a production stack.

3 · The rates are illustrative. $3/MTok in, $15/MTok out, reads at 0.1× — placeholders,
labelled as such in costs.Rates. Provider prices change.

What does transfer, and is worth bringing to a client:

• The line-item structure of a per-query cost. Turning "AI is expensive" into cached prefix
  / evidence / question / generation / rerank / embedding is the conversation, and it is one
  most vendors cannot have.
• The levers in order, with the quality cost of each named. The first three are free; the
  last two are trades you declare out loud.
• The method: build the measurement first, change one thing, report the interval.
• The index-time versus query-time argument. Index-time compute is paid once, query-time
  forever — that reframing changes architecture decisions and costs nothing to explain.

How to actually do it: put the client's real rate card into costs.Rates(...), their real
k, and their measured tokens-per-chunk. unit_economics() then produces their table. That
takes ten minutes and is defensible; quoting our number is not.

Worth saying plainly: a client who catches you quoting a number from a teaching corpus will
discount everything else you said. The honest version is also the one that survives the
follow-up question.

---

_{Seeded by faculty as a worked example of the format. Start your own thread rather than replying here unless you have something to add.}

[akash-coded]

Marcus · cohort peer

Contrarian view: is this not over-cautious? The cost model in notebook 07 uses published per-token prices and real token counts. The arithmetic is arithmetic. If a client asks "roughly what does a query cost", saying "about a cent" is more useful than refusing to answer.

I understand not quoting the recall numbers. But cost seems different in kind.

[akash-coded]

Dana · reviewer · owns the eval harness

The distinction you are drawing is real and I want to agree with half of it, then show you where the half that is wrong actually bites.

You are right that cost and quality are different in kind. Quality numbers are corpus-dependent in a way that makes them nearly meaningless off this corpus. Token counts are more portable — a 4,000-token prompt is 4,000 tokens anywhere.

Where it breaks is that the per-query price is not the number a client is asking for. They are asking what the system costs, and the query price is typically a minority of it. Off notebook 07's own model, for a mid-size deployment:

                              share of monthly bill
generation tokens                      31%
embedding — initial backfill           18%   (one-off, but it lands in month one)
embedding — re-embed on drift          14%   (recurring, always forgotten)
vector store / OpenSearch cluster      22%   (fixed, does not scale down)
reranker inference                      9%
trace + eval storage                    6%

Quote "about a cent a query" and you have described 31% of the bill with high confidence and implied the other 69% is rounding. That is the conversation that goes badly in month three, and it goes badly because the number you gave was accurate.

What is safe to say, and it is genuinely useful:

"The per-query token cost is around X, and I can show you the model. It is roughly a third of the total — the cluster and the re-embedding cycle dominate, and both depend on facts about your corpus I do not have yet. Give me your document count and change rate and I will turn this into a real number in a week."

That is more credible than a confident cent-per-query, not less. The methodology transfers; the constants do not — and saying which is which is the thing that makes a client trust the next number you give them.

[akash-coded]

Marcus · cohort peer

The 31% breakdown is the argument. I was going to quote a number that was true and load-bearing for the wrong conclusion.