[maths] Should I normalise my embeddings? What actually breaks if I do not?

[akash-coded]

Casebook thread. A reconstructed conversation, written for teaching. The people in it
are illustrative; every number is real and reproducible from the notebook cited below.
Replies are welcome — if you hit this yourself, add your numbers.

The question in one line

Every tutorial normalises embeddings before storing them and none of them say why. Is it just so
cosine and dot product agree, or is something else going on?

What I have already tried

Stored both normalised and un-normalised copies. Rankings differ, but I cannot tell which is
right, and I do not know what I would be giving up either way.

Where

nanorag/store.py — exact_vector()

[akash-coded]

Priya · cohort peer

The short answer is that it is only so cosine and dot product agree — if ‖v‖ = 1 the
denominator in cosine is 1 and the two are identical. Once they agree you can use the cheaper
one, which is dot product, and every ANN library is optimised for it.

So: normalise, use dot product, move on.

[akash-coded]

Rafael · cohort peer · came from search infra

That is true and it is the half that does not matter much. There is a second half, and it is
the one that bites.

Normalising throws magnitude away, and magnitude is not always noise. For a lot of encoders
‖v‖ correlates with something real — document length, or how confidently the model placed the
text. Two consequences:

• If magnitude tracks length, normalising removes your only length signal in the dense leg.
  BM25 has b for exactly this reason; the dense leg silently has no equivalent.
• Some encoders emit near-zero-norm vectors for input they cannot represent — boilerplate,
  tables, code. Normalising a near-zero vector amplifies whatever noise it contains to unit
  length, and that document now competes on equal terms with everything else.

Worth doing once on your own corpus: histogram ‖v‖ before normalising and read the bottom
percentile. It is usually the junk.

[akash-coded]

Marking this because the question underneath — what breaks if I do not — has a sharper answer
than either half so far.

If you keep magnitude, you are doing MIPS, and MIPS is not a metric space.

Inner product fails the requirements outright: q·q = ‖q‖² is neither zero nor minimal, and
there is no triangle inequality. Every tree- and pivot-based index — ball trees, k-d trees,
anything that prunes using the triangle inequality — is not slower under MIPS, it is wrong.
It prunes branches that contain the answer, and it does so silently.

The standard escape is a lift that turns MIPS into ordinary nearest-neighbour search by adding
one dimension that absorbs the magnitude difference, so every document lands on a common sphere.
Full derivation in Cosine, dot product and MIPS.

The practical answer to your actual question: normalise, unless you have measured that
magnitude carries signal you want. But whichever you choose, know which one your stack is
doing — teams routinely do not, and the check is one line: are the stored vectors unit norm?

And the failure mode to fear. Normalise at index time and not at query time, or use one
library's default and not another's, and you get a system that runs, returns plausible results,
and is quietly mis-ranked. Nothing errors. It is the same shape as a mixed-embedder_tag index:
cosine returns well-formed numbers for vectors that mean nothing to each other.