fix(search): align corpus ranking and MCP retrieval contracts

[morluto]

Description

Corpus search mixed several ranking models: repository results used a handwritten score, thread and hydrated-facet ranks came from separate FTS indexes, exact repository filters bypassed normal query matching, and combined search compared ranks that had no shared scale. MCP exposed those inconsistencies through a large overlapping catalog and sometimes returned details or coverage from a different read than the ranked result.

This change gives each searchable corpus a single retrieval contract:

Corpus	Weighted FTS5 document
Repositories	owner/name 10, topics 5, description 2
Threads	title 10, labels 5, body 2, hydrated facets 0.5
Code	path 5, content 1

Thread headers and bounded hydrated evidence are materialized into one atomic search document, so one BM25 invocation determines rank. Hydrated replacements update that document only after a complete retrieval, and results distinguish retained hydrated evidence from thread text while reporting truncation. Relevance is the default order, source freshness breaks equal-rank ties, and sort=updated provides explicit newest-first retrieval. Combined search all is rejected because BM25 ranks from different indexes are not comparable.

Repository and code search now return bounded matching excerpts. explain_match reads the exact FTS evidence for repository topics and descriptions, thread fields and facets, and exact code documents, so its snippet cannot silently omit a match outside an arbitrary content prefix.

Code snapshots persist an indexing manifest and return selected-snapshot coverage in the same SQLite read transaction as matches and totals. Re-indexing the same commit atomically replaces both its manifest and child documents while preserving the commit's original ordering; a failed replacement rolls back to the prior searchable snapshot. Legacy snapshots report unknown coverage instead of implying that zero recorded skips means complete coverage.

The MCP surface is narrowed around those contracts:

• capability requirements are declared with each tool instead of maintained in a second pruning inventory;
• broad reader/operator interfaces are split into the capabilities each tool actually uses;
• capability-based toolsets and --read-only mode advertise only available operations;
• the default contribute catalog stays below a 96 KiB serialized context budget with measured headroom;
• concise responses are the default for live search and job polling;
• validation accepts managed workspace IDs instead of arbitrary host paths;
• repository dossiers are persisted explicitly instead of being rebuilt during reads;
• batch duplicates, unsupported resource aliases, inexact totals, and unstructured tool errors are rejected or reported directly;
• deterministic agent-evaluation fixtures cover relevance versus freshness, README/code-index boundaries, and persisted-only reads.

Existing corpora require the explicit, backed-up gitcontribute corpus migrate --yes flow to advance from schema 1 to schema 4. Corpus reads remain offline, GitHub access remains an explicit read capability, and no GitHub mutation path is added.

Suggested review order

1. internal/corpus/migrations/, search.go, and repository_search.go for unified FTS documents, weighting, canonical repository queries, evidence excerpts, and atomic facet replacement.
2. internal/corpus/code.go and its tests for snapshot-consistent results, same-commit replacement, coverage, and totals.
3. internal/app/search.go and mcp_v1.go for ordering, cursor scope, exact explanations, and removal of cross-index ranking.
4. internal/mcpserver/catalog.go, server.go, and the capability-specific files for toolsets, interfaces, schemas, annotations, and output bounds.
5. MCP/application boundary tests, agent-evaluation fixtures, and architecture documentation.

The branch is split into ten semantic commits following that progression: corpus projections, MCP contracts, application wiring, tests/docs, read cleanup, adapter splits, capability binding, query/retry correctness, exact evidence handling, and the final SQL row-lifecycle correction found by CI.

Testing

• make verify
• make test-race
• Regression coverage for weighting, newest-first sorting, canonical owner/repo queries, cursor scope, atomic hydrated evidence, same-commit code re-indexing and rollback, truncation attribution, exact explanation excerpts, code-index coverage, exact totals, tool discovery, read-only mode, and managed workspace paths

Compatibility and scope

• Storage and side-effect invariants remain intact
• No unrelated cleanup or generated-output churn is included

Compatibility changes are intentional: combined cross-index search is removed, the default MCP catalog is smaller, specialized tools require their toolset, and existing corpora require an explicit migration before use with this schema.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.