Standardize discovery pagination and grouped metadata

[Widthdom]

Summary

• standardize bounded discovery responses with opaque query- and generation-bound cursors, authoritative totals, stable-at metadata, byte limits, and resumable partial scans
• bind outline and unused pagination to their effective scope, filters, generated-code policy, ordering, and index snapshot
• make grouped search completion metadata authoritative and preserve native compact aggregate contracts
• keep English and Japanese CLI documentation aligned with the implemented contracts

Root cause and impact

Several discovery surfaces exposed partial or grouped results without enough state to resume safely or distinguish authoritative totals from lower bounds. Some cursor fingerprints omitted effective filters, while aggregate modes could be projected as row output. The updated contracts prevent stale or mismatched cursor reuse, duplicate/omitted pages, lost aggregate payloads, and misleading counts.

Validation

• dotnet build CodeIndex.sln -c Release -p:UseSharedCompilation=false --no-restore — passed with 0 warnings and 0 errors
• final focused Release/net8 regression set — 9 passed
• envelope-focused Debug/net8 set — 36 passed; selector-focused set — 1 passed
• full Release/net8 suite — 10,231 passed, 7 skipped, 1 non-deterministic console-isolation failure; isolated retry passed 1/1 and is tracked by Isolate request-id logging from console-sensitive validation tests #4798
• full Release/net9 suite — 9,771 passed, 404 skipped, 1 existing suite-only timeout tracked by Full-suite Console.Error capture can leave a disposed global writer #4749; isolated retry passed 1/1
• dotnet format CodeIndex.sln --verify-no-changes --no-restore — passed
• dotnet run --project tools/CodeIndex.Changelog -c Release --no-build -- check — validated 42 fragments
• full CodeIndex rebuild and status --check --json — 1,002 files, 42,027 symbols, 347,959 references; fresh with 0 warnings/errors
• Codex adversarial review — two workflow-limited rounds completed; all six actionable findings were addressed with regression coverage

Documentation and changelog

• updated the English and Japanese sections in USER_GUIDE.md, DEVELOPER_GUIDE.md, and TESTING_GUIDE.md
• added changelog.d/unreleased/4730.fixed.md
• added changelog.d/unreleased/4734.fixed.md
• added changelog.d/unreleased/4735.fixed.md

Related follow-up

• Isolate request-id logging from console-sensitive validation tests #4798

Fixes #4730
Fixes #4734
Fixes #4735