Skip to content

Add --compact JSON output mode to entire search (ENT-1527) - #1908

Open
evisdren wants to merge 8 commits into
mainfrom
evis/ent-1527-search-json-compact
Open

Add --compact JSON output mode to entire search (ENT-1527)#1908
evisdren wants to merge 8 commits into
mainfrom
evis/ent-1527-search-json-compact

Conversation

@evisdren

@evisdren evisdren commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • entire search --json embeds each checkpoint's full prompt, so a default page runs 10–36KB — the dominant token cost for AI-agent consumers (see ENT-1527).
  • Adds --compact (implies --json, rejected with --code): per hit only id, type, repo (org/name), branch, author, date, filesTouched, rerank score, and a whitespace-collapsed title snippet truncated to 200 runes — never the full prompt. Envelope (total/page/total_pages/limit/counts) matches the full JSON mode.
  • Agents fetch full detail for the one hit they care about via entire checkpoint explain <id>. Extracted a shared paginateSearchResults helper so both JSON writers use the same client-side pagination.

Test plan

  • mise run fmt && mise run lint — clean
  • Unit + integration tests (mise run check)
  • E2E canary (Vogon + roger-roger) — all passed
  • New tests: compact output shape (full prompt absent), long-prompt title truncation, --compact + --code rejection

🤖 Generated with Claude Code


Note

Low Risk
CLI-only output formatting and flag validation; no auth, API, or data-path changes.

Overview
Adds --compact to entire search and entire checkpoint search for agent-friendly JSON. It implies --json and emits a slim per-hit shape (id, type, repo, branch, author, date, filesTouched, score, truncated title) instead of full checkpoint prompts; --compact with --code is rejected.

Pagination logic is factored into paginateSearchResults, shared by full and compact JSON writers. Help and examples document the new flag; tests cover compact shape, long-title truncation, and the --code incompatibility.

Reviewed by Cursor Bugbot for commit 3888ec4. Configure here.

@evisdren
evisdren requested a review from a team as a code owner August 6, 2026 04:07
Copilot AI lite review requested due to automatic review settings August 6, 2026 04:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an agent-friendly --compact JSON output mode to entire search / entire checkpoint search, reducing per-hit payload size by omitting full prompts while keeping the existing JSON envelope and pagination behavior consistent.

Changes:

  • Added --compact flag (implies --json, rejected with --code) and routed JSON output to a new compact writer.
  • Factored client-side pagination into a shared paginateSearchResults helper used by both JSON writers.
  • Added tests covering compact shape (no full prompt), title truncation, and --compact + --code validation; updated help examples.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
cmd/entire/cli/search_cmd.go Adds --compact flag, shared pagination helper, and compact JSON writer that truncates title snippets and excludes full prompts.
cmd/entire/cli/search_cmd_test.go Adds unit tests for compact JSON output and flag incompatibility behavior.
cmd/entire/cli/checkpoint_group.go Updates canonical checkpoint search help examples to include --compact.

Comment thread cmd/entire/cli/search_cmd_test.go
evisdren and others added 7 commits August 6, 2026 11:16
Agents paid 10-36KB per search result page because every hit embedded
the checkpoint's full prompt. --compact (implies --json) trims each hit
to id, type, repo, branch, author, date, filesTouched, rerank score, and
a 200-rune title snippet; full detail stays one `entire checkpoint
explain <id>` away.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZAM1T0QT3PG3639FX0HKWWA
…ll-down

Compact scanning plus a single `entire checkpoint explain <id>` on the
winning hit is the token-cheap workflow ENT-1527 enables; the skill now
teaches it (re-search narrower rather than mass-explain).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZAMPZWGPJVPSM1N3FAGEBQT
Repo and pr result types (reachable via --all-repos) have no typed
struct, so the Result accessors returned "" and a compact hit collapsed
to just {id, type, score}. Add rawData fallbacks (mirroring the existing
ResultID fallback) for title, repo, org, author, and createdAt so agents
can tell what matched. Also drop the unnecessary ENTIRE_CODE_SEARCH env
tweak from the --compact/--code rejection test so it can run in parallel.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZANMNE5W7QA2Q02SDMZBCBR
Address trail review findings on the compact search PR:

- Gate the rawData accessor fallback on unknown types: UnmarshalJSON sets
  rawData for every row, so typed rows (e.g. a session with an empty
  displayName) started surfacing raw payload fields that their typed
  accessors deliberately suppress — affecting the TUI, not just --compact.
  The payload is now decoded once at unmarshal time, only for repo/pr
  rows, into a cached map (also removes the per-render re-decode).
- Add the missing branch/headRefName fallback so PR hits carry a branch.
- Keep Meta.Snippet and Meta.MatchType in the compact shape: the snippet
  is the matched text that lets an agent pick which hit to explain.
- Scope the search skill's drill-down step: explain <id> for checkpoint/
  commit hits in the current repo, --session <id> for session hits, and
  compact fields alone for repo/pr or cross-repo hits, which explain
  cannot read.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZC0CYB5M08TJ4HBD67TXP4P
fullName is "owner/repo" throughout the codebase, but ResultRepo's raw
fallback returned it whole while ResultOrg separately returned org, so
org+"/"+repo joins (compact writer, TUI meta line and static table)
produced strings like acme/acme/backend. ResultRepo now always returns
a bare name — falling back to fullName's repo segment — and ResultOrg
learns fullName's owner segment when no org key is present.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZC115MRKT470GNER4MJ389F
explain --session filters the checkpoint list on the current branch; it
is not a session detail view and returns nothing useful for hits from
other branches or repos. Reword the skill's drill-down step to say so
and route those hits to the compact fields instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZC2DWFRPHZZHJHKC9HFNC9F
Sessions are projected from checkpoints, not indexed, so the skill now
routes all drill-down through checkpoints: compact scan, then
`entire checkpoint explain <id>`, then `--full` for the session
transcript — dropping the `--session` branch entirely. Search help and
the compact writer docs mention the `--full` escalation too.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 01KZC4477M7R14J952W4XAQMBH
@evisdren
evisdren force-pushed the evis/ent-1527-search-json-compact branch from b923840 to 218aa9b Compare August 6, 2026 18:18
Entire-Checkpoint: 01KZCP4STDFDV8EBT4SK2A6E87
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants