docs: findings on relationship-discovery quality — the missing floor under b2 similar

[samkeen]

An assessment doc, not a design ruling: why the always-ten-cards behavior
lives in discover::candidates by design (not in the #137→#144 search
plumbing, which stays), why the reranker issue #28 explicitly excludes
similar, why the eval's rank-only discovery metrics are structurally blind
to precision, and the recommended path — rule the posture change in the
design docs, extend the eval with negative anchors + score distributions,
ship a model-relative quality floor calibrated from that data, and escalate
to a discovery-side pair-scorer only if the measured score piles overlap.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01T7kdSfpfyUasBo9D9fN2TN

Summary by CodeRabbit

• Documentation
  • Added comprehensive findings on discovery result quality and candidate ranking.
  • Documented how result limits, backfilling, score thresholds, and model-specific scoring affect displayed recommendations.
  • Added recommendations for improved evaluation metrics, negative examples, score calibration, and clearer empty states.
  • Recorded potential future use of discovery-specific reranking if quality thresholds prove insufficient.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 900b8ab1-0a47-4ad4-bda4-f8eb071dbe98

📥 Commits

Reviewing files that changed from the base of the PR and between f0108f7 and ee6b110.

📒 Files selected for processing (1)

• docs/discovery-quality-findings.md

---

📝 Walkthrough

Walkthrough

The PR adds a discovery-quality findings document for b2 similar. It explains candidate generation, evaluation gaps, threshold calibration, raw score display, and an ordered remediation plan.

Changes

Discovery quality

Layer / File(s)	Summary
Discovery quality findings and remediation plan docs/discovery-quality-findings.md	Documents candidate oversampling, torn-read backfilling, unlinked-note ranking, and result truncation. It proposes negative anchors, score-distribution metrics, a model-relative quality floor, honest empty states, and a conditional discovery reranker. It leaves existing search plumbing and graph-distance reordering unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• AlteredCraft/B2#39: Modifies discover::candidates and its UI flow for b2 similar discovery.
• AlteredCraft/B2#61: Introduces discovery behavior and evaluation metrics analyzed by this PR.
• AlteredCraft/B2#71: Relocates discovery and evaluation design references and harness metadata.

Poem

A rabbit reads the findings with care,
Ten loose candidates hop through the air.
Anchors and thresholds now mark the way,
Empty states wait when scores cannot stay.
“Measure first,” says the rabbit, “then rerank!”

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the document's findings on relationship-discovery quality and the proposed quality floor for b2 similar.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/b2-relationship-detection-quality-xkopq0

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}