The retrieval opinion you hold that most people here would disagree with #133
akash-coded
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
One per reply. Argue for it in a paragraph. Bring a number if you have one — but an opinion
you cannot yet support is still worth posting, because someone may have the measurement.
To start, three of mine, all of which I have been argued with about:
1 · Most reranking is an expensive identity function. If the reranker's features overlap the
first stage's, it re-sorts by information the pipeline already used. This repo measured one making
retrieval worse at every k. The general rule is in The Reranker
Mirror.
2 · Growing the eval set beats almost every engineering option, and nobody does it. A day of
generator work moves more decisions than a month of tuning, because it changes what you can see.
Every argument about whether a 2-point delta is real is an argument about
n.3 · Chunking strategy is over-discussed and analyzer configuration is under-discussed. People
will spend a week on semantic chunking and never check whether their tokenizer splits
ERR_CONN_RESETinto three terms that appear in every document. One of those is worth a week.The rule for this thread
Disagree with the claim, not the person, and prefer evidence to seniority. A thread where the
loudest person wins is a thread nobody learns from.
All reactions