feat(jrag): implement --count/--exists/--fields output flags (#376)#422
Merged
Conversation
HumanBean17
added a commit
that referenced
this pull request
Jul 10, 2026
Follow-up to 37c5bc3 (PR #422) after a 4-reviewer fan-out. - render: _project_to_fields now delegates to project_envelope() and only rewrites `nodes` on the already-copied result, so the per-field Envelope construction can't drift out of sync as Envelope fields are added (was a 12-field duplication of project_envelope's field list). - render: a whitespace/comma-only --fields allowlist now falls back to the normal projection instead of stripping every node to {}. - docs: "(every command)" was factually wrong — status/microservices/ vocab-index reject these flags at parse time. Reworded to call that out (it contradicted the "inapplicable flags never silently ignored" principle). Also documented the --exists json form and noted --fields is ignored with --count/--exists. SKILL.md + explorer-rag-cli.md resynced to install_data. - tests: pin the previously-untested precedence links (count>fields, exists>fields), --fields CSV whitespace/dedupe/empty handling, and the --exists json shape on non-ok status. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Jul 11, 2026
Open
Add three output-shaping flags to the jrag common parser, available on every
listing, locate, traversal, orientation, and search command:
- --count prints just the result count (bare int in text; {"status","count"}
in json). Shape-aware: nodes for listings, edges for traversals, 1 for
inspect.
- --exists prints true/false and exits 0 on a hit / 2 on a miss, including
resolve misses (scriptable existence gate: find/inspect X --exists).
- --fields fqn,role,... projects each node to a comma-separated field
allowlist that overrides --detail (primarily a --format json lever).
--brief is intentionally NOT implemented: --detail brief already provides
identity-only output end-to-end, so a separate flag would be a redundant
alias that muddies the detail-orthogonal-to-format model.
Implementation: render() gains count/exists/fields kwargs (exists takes
precedence over count); a single new _emit() funnel routes every command's
success/not_found path and drives the --exists exit code via the shared
count_results/has_results helpers. _core_parser commands (status /
microservices / vocab-index) are excluded -- vocab-index doesn't render an
envelope, so the flags would be silently ignored there.
Re-advertised in skills/explore-codebase-cli/SKILL.md and
agents/explorer-rag-cli.md; re-synced to install_data.
Co-Authored-By: Claude <noreply@anthropic.com>
Follow-up to 37c5bc3 (PR #422) after a 4-reviewer fan-out. - render: _project_to_fields now delegates to project_envelope() and only rewrites `nodes` on the already-copied result, so the per-field Envelope construction can't drift out of sync as Envelope fields are added (was a 12-field duplication of project_envelope's field list). - render: a whitespace/comma-only --fields allowlist now falls back to the normal projection instead of stripping every node to {}. - docs: "(every command)" was factually wrong — status/microservices/ vocab-index reject these flags at parse time. Reworded to call that out (it contradicted the "inapplicable flags never silently ignored" principle). Also documented the --exists json form and noted --fields is ignored with --count/--exists. SKILL.md + explorer-rag-cli.md resynced to install_data. - tests: pin the previously-untested precedence links (count>fields, exists>fields), --fields CSV whitespace/dedupe/empty handling, and the --exists json shape on non-ok status. Co-Authored-By: Claude <noreply@anthropic.com>
HumanBean17
force-pushed
the
feat/jrag-output-flags-376
branch
from
a466dab to
32db2f3
Compare
HumanBean17
added a commit
that referenced
this pull request
Jul 11, 2026
…goldens Rebase onto master (#422 --count/--exists/--fields; #430 find/rendering QA) left the 4 refactored read handlers (inspect/callers/callees/flow) routing their resolve-miss through ``print(render(pe.env)); return pe.rc``, bypassing master's new ``_emit`` funnel — so ``<cmd> Missing --exists`` returned rc 0 instead of 2 and ignored the flag. Switch them to ``_emit(pe.env, args)``, mirroring master's ``_resolve_traversal_node`` / inspect resolve-miss path, and drop the now-unused ``render`` imports. find (kind-guard) and search (NodeFilter validation) keep ``print(render)`` — master intentionally bypasses ``_emit`` for usage errors so the actionable message isn't hidden by a count/exists shape. Master #430 also enriched route/symbol node rendering (name/method/path/ framework/resolved/file on routes; name/file on symbols), so regenerate the 3 affected byte-identity goldens (callers_route, flow_merge, find_filter); the other 8 goldens are byte-identical. Cold==daemon byte-identity re-verified (tests/watch 116, incl. all 6 IPC goldens). Co-Authored-By: Claude <noreply@anthropic.com>
HumanBean17
added a commit
that referenced
this pull request
Jul 11, 2026
* test(watch): characterize Lance atomic versions, graph COW snapshot, warm model reuse
Three heavy-gated characterization tests (JAVA_CODEBASE_RAG_RUN_HEAVY=1) that
lock in the concurrency pillars the `jrag watch` daemon assumes (design §4.7/§9):
- test_lance_atomic_versions: a fresh lancedb.connect after a cocoindex catch-up
sees a strictly newer atomic version with the new file's chunks fully present;
a handle opened before the write keeps a consistent (pre-write) snapshot.
- test_graph_cow_snapshot (load-bearing): a read-only reader on a file COPY of
code_graph.lbug is isolated from run_incremental_graph writing the original
(returncode 0, no single-writer conflict); the sidecar keeps the pre-write
node count while the original grows.
- test_warm_model_reuse: _get_sentence_transformer returns one process-wide
singleton, reused across calls and by search_v2.
Key characterization findings recorded in docstrings:
- Lance version delta per catch-up is ~+4 (catch-up + serialized optimize), not
+1; the pillar only needs monotonic atomic growth.
- LadybugGraph.meta()["counts"] is NOT a reliable node count across an incremental
rebuild (phantom resolution offsets new nodes: summed total 88->88 while the
real :Symbol count grew 88->93); tests use a direct MATCH query instead.
- LadybugGraph.reset_for_path does not exist yet (Task 6 adds it); singleton is
reset manually as the existing heavy e2e tests do.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add runtime socket/pid/state path derivation
* fix(watch): idiomatic imports + test isolation in paths.py
* feat(watch): add project pidfile + flock
ProjectLock: exclusive per-project lock keyed on paths.pid_path(index_dir).
acquire opens the pid file r+/w+, takes flock(LOCK_EX|LOCK_NB), translates
BlockingIOError to LockHeldError(stored_pid, path), writes our pid on success.
release closes the handle (drops the flock) and unlinks the pid file iff it
still names us. is_holder + classmethod read_holder (stale-pid aware).
WatchUnsupportedPlatform raised on construction when fcntl is unavailable.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add IPC request/response contract + NDJSON codec
* fix(watch): export error-kind constants + clean up protocol tests
* refactor(jrag): extract payload cores for read commands (behavior-preserving)
Extract payload-returning cores for the callers/callees/flow/search/find/inspect
read commands into a new read_payloads.py, so the upcoming jrag watch daemon
can serve them over a socket without touching rendering. Each handler becomes
'payload = <cmd>_payload(args, cfg, graph); render(payload, args)' — rendering
byte-identical to before.
- callers_payload/callees_payload/flow_payload carry the full assembly verbatim,
including the three ad-hoc folds: callers EXPOSES-inbound, callees CLIENT-role
HTTP_CALLS, and flow inbound/outbound merge.
- search_payload/find_payload/inspect_payload wrap the *_v2 (or
find_by_name_or_fqn) backend call.
- Resolve/guard/validation failures raise PayloadError(env, rc); the handler
renders the carried Envelope, so error output is byte-identical too.
New golden-output equivalence tests (tests/jrag/test_read_payloads.py +
tests/jrag/golden/) prove byte-identity for 9/10 commands; inspect is pinned by
canonicalized JSON because its edge_summary dict key order is pre-existing
non-deterministic in describe_v2 (verified on unmodified HEAD). All three folds
are exercised by the golden fixtures.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add WarmResources + graph COW snapshot lifecycle
WarmResources holds the warm embedding model and read-only LadybugGraph
reader for the jrag watch daemon, and owns the copy-on-write graph snapshot
lifecycle: begin copies code_graph.lbug to a .lbug.snapshot sidecar and
serves reads from it while a reindex subprocess writes the original; commit
drops the sidecar reader and reopens the updated original. Adds
LadybugGraph.reset_for_path classmethod (clears the cached singleton on None
or matching path) so WarmResources can force the next get() to reopen.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(watch): harden reset_for_path path resolution + commit_graph_snapshot cleanup
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add Unix-socket server + per-command dispatch
WatchServer binds an AF_UNIX socket (chmod 0o600), runs an inline accept loop
over newline-delimited JSON requests (Task 4 protocol), dispatches each to its
Task-5 payload core, and writes the encoded response. dispatch maps errors per
the pinned IPC contract (_IndexStale -> stale_index, ProtocolMismatch/ValueError
-> bad_args, any other Exception incl. PayloadError -> backend_error, unknown
cmd -> unknown_command). serialize() uses model_dump(mode="json") + recursion
(the cold path's --format json Envelope emit is tangled with handler-specific
projection; the daemon ships the serialized payload and the client renders).
15 transport/dispatch tests (stub warm + stub PAYLOAD_FNS).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add IPC client + cold fallback in read commands
watch/client.py implements the try-daemon-then-cold-fallback seam:
is_daemon_alive, request (AF_UNIX, 2s timeout, ProtocolMismatch->cold),
DaemonUnavailable/DaemonError, and get_payload (the single seam each read
handler calls). On daemon success the payload is reconstructed into the same
object the cold core returns (model_validate for the pydantic *Output models;
SymbolHit(**row) rebuild for find query-mode rows); on DaemonUnavailable /
DaemonError it cold-falls-back to the identical <cmd>_payload core via
_load_graph (a cache hit on the already-loaded singleton).
The 6 jrag read handlers (search/find*/inspect/callers/callees/flow) acquire
their payload via get_payload("<cmd>", vars(args), cfg, cold_core=<cmd>_payload);
only the payload-acquisition line + a lazy import per handler changed -- render
and rc logic are untouched, so the cold path stays byte-identical (Task 5 golden
suite 15/15 unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add debounced per-type reindex watcher + watchdog dep
SourceWatcher observes the source tree via a watchdog backend (native Observer
with a polling fallback for 'auto'; explicit 'watchdog'/'polling') and, after a
single-thread debounce window, runs the minimal reindex for the changed types:
java -> vectors THEN graph (graph subprocess under WarmResources' COW snapshot,
begin before / commit always in a finally); sql/yaml -> vectors only. Status
flows only through on_event callbacks. Adds watchdog>=6,<7.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(watch): monkeypatch fixture in watcher tests + document target/ firing + pair begin/commit
* feat(config): add watch: YAML block (debounce_ms, backend, poll_interval_ms)
Parse the new optional `watch:` block in resolve_operator_config into three
ResolvedOperatorConfig knobs (debounce_ms, backend, poll_interval_ms) plus
companion *_source fields. Precedence: CLI flag > YAML > built-in default (no
env vars introduced). Inline floors/validation: debounce_ms >= 100,
backend in {auto,watchdog,polling}, poll_interval_ms >= 200 — out-of-range
values fall back to the default with a one-line stderr warning.
_pick_int gains an optional cli_val kwarg (default None) mirroring _pick_str,
so the CLI-override tier flows through the helper; existing callers untouched.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(watch): add jrag watch foreground/detach/stop/status lifecycle
Task 11 (capstone): assemble the watch components into a daemon process and
expose the `jrag watch` subcommand with its four lifecycle verbs.
- watch/daemon.py `WatchDaemon`: acquires the project lock, eagerly warms the
model (fail-fast), installs SIGINT/SIGTERM handlers, starts server+watcher,
writes a state file for cross-process status, renders a rich Live panel (TTY)
or a one-line startup (non-TTY), and tears down watcher->server->lock on stop.
The serving-path shutdown ends with os._exit(0) (mirrors _console_script_main)
to dodge the pyarrow/lance worker-thread SIGABRT once a search has loaded
lancedb in-process. Early-failure paths (lock held / model load) return 2
normally (no lance threads yet).
- jrag.py: `watch` subparser (parents=_core_parser, no auto-scope) with
--detach/--stop/--status/--debounce-ms/--backend; `_cmd_watch` + helpers.
--status/--stop read ProjectLock.read_holder only (never acquire the lock).
--detach spawns `python -m java_codebase_rag.jrag watch` detached (setsid +
redirected stdio) and waits for is_daemon_alive. `--debounce-ms`/`--backend`
plumbed through _resolve_cfg.
- tests/watch/test_daemon.py: 10 lifecycle tests (stubbed warm/watcher in a
subprocess so the real os._exit shutdown runs outside pytest) + a heavy-gated
golden IPC test proving `jrag search` over the socket is BYTE-IDENTICAL to the
cold path, with a _load_graph spy proving the hot path was actually served.
Two integration fixes found + resolved by the tests:
1. Stale socket before bind: the daemon acquires the lock BEFORE server.start,
so the server's `read_holder is None` stale-socket guard is inert (it sees
its own pid). The daemon now clears its own stale socket before bind (it
holds the exclusive lock, so it owns the path).
2. `python jrag.py` as a script shadows stdlib `ast` with java_codebase_rag.ast
(inspect -> ast.NodeVisitor crashes). Detach now invokes the daemon via
`python -m java_codebase_rag.jrag`.
queries_served left at 0 for v1 (wiring a count out of WatchServer is out of
this task's commit surface and the brief marks it optional).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(watch): lazy daemon import in _cmd_watch + drop queries_served panel + detach fail-fast
* docs(watch): document jrag watch + watch: config across CLI/CONFIGURATION/ARCHITECTURE/DESIGN + skill tip
- JAVA-CODEBASE-RAG-CLI.md: new `### jrag watch` subsection (four modes,
--debounce-ms/--backend flags, cold-fallback guarantee, Unix-only note)
- CONFIGURATION.md: `watch:` YAML block (debounce_ms/backend/poll_interval_ms
with defaults + floors) in the §2 reference
- ARCHITECTURE.md: watch daemon in the read path + a watch-path note
(warm resources, subprocessed reindex, graph COW snapshot, Lance atomic
versions, pidfile/flock); watch/ in the layout table + extension points
- DESIGN.md: `jrag watch` as a first-class surface + the watch non-goals
(Unix-only; not a boot service; not multi-project; cold path unchanged;
not for the MCP surface)
- skills/explore-codebase-cli/SKILL.md + agents/explorer-rag-cli.md: tip to
run `jrag watch` once per session for fast, fresh queries
- tests/test_docs_watch.py: docs-accuracy test pinning the operator surface
Co-Authored-By: Claude <noreply@anthropic.com>
* docs(config): correct watch.poll_interval_ms comment (polling-only, not watchdog-only)
* fix(watch): synchronize COW snapshot flip + E2E golden byte-identity for all read cmds + doc limitations
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(watch): carry find --fuzzy payload keys through IPC client
Rebase onto master pulled in PR #413 (find --fuzzy exact→prefix→contains),
which added ``matched_mode``/``identifier_matched`` to the find query-mode
payload (read_payloads.find_payload). The server serializes the whole dict,
but the watch IPC client reconstructs find query mode with a hardcoded key
set that predates --fuzzy — so the daemon path would KeyError in
``_render_find_query``. Thread the two keys through the reconstruction and
the client unit-test mock. Cold path was already correct; proven by the
find_query golden byte-identity test over the real socket.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(watch): adopt _emit in refactored resolve-miss paths + regen #430 goldens
Rebase onto master (#422 --count/--exists/--fields; #430 find/rendering QA)
left the 4 refactored read handlers (inspect/callers/callees/flow) routing
their resolve-miss through ``print(render(pe.env)); return pe.rc``, bypassing
master's new ``_emit`` funnel — so ``<cmd> Missing --exists`` returned rc 0
instead of 2 and ignored the flag. Switch them to ``_emit(pe.env, args)``,
mirroring master's ``_resolve_traversal_node`` / inspect resolve-miss path,
and drop the now-unused ``render`` imports. find (kind-guard) and search
(NodeFilter validation) keep ``print(render)`` — master intentionally bypasses
``_emit`` for usage errors so the actionable message isn't hidden by a
count/exists shape.
Master #430 also enriched route/symbol node rendering (name/method/path/
framework/resolved/file on routes; name/file on symbols), so regenerate the 3
affected byte-identity goldens (callers_route, flow_merge, find_filter); the
other 8 goldens are byte-identical. Cold==daemon byte-identity re-verified
(tests/watch 116, incl. all 6 IPC goldens).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(ci): sync install_data, heavy-gate golden test, skip watch on Windows
Three CI failure classes after the rebases onto master:
1. install_data out of sync (all platforms). The deployed skill/agent copies
(src/.../install_data/{skills,agents}/explorer-rag-cli) lacked the
``jrag watch`` tip added to the dev source — test_install_data_sync caught
the drift. Re-run scripts/sync_agent_artifacts.py (never hand-patch
deployed copies).
2. test_golden_output_byte_identical cross-platform fragility (callees_symbol /
flow_merge on linux+windows; search on intel-mac). The e2e golden builds a
real index and byte-compares CLI stdout captured on one platform, but
graph-traversal edge order follows indexing file-traversal order (OS-
dependent) and the search golden relies on the Lance table being absent
(index-dir nondeterminism). Gate it behind JAVA_CODEBASE_RAG_RUN_HEAVY=1
(same idiom as test_lancedb_e2e) — it's a LOCAL regression guard, not a
portable CI check. CI still covers the cold read path via tests/package/*;
warm==cold byte-identity is covered (heavy) by tests/watch/test_daemon.py.
The in-process Layer 2 tests stay in CI (only the e2e golden is gated).
3. tests/watch on Windows. jrag watch is Unix-only by design (AF_UNIX sockets
+ fcntl flock; watch/lock.py raises WatchUnsupportedPlatform). The tests
exercise those primitives directly, so add tests/watch/conftest.py that
skips the whole tree when fcntl is unavailable (the product's own gate).
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements three output-shaping flags from #376 on the
jragcommon parser (every listing / locate / traversal / orientation / search command).--briefis intentionally skipped — see below.--count{"status","count"}(json). Shape-aware: nodes (listing) / edges (traversal) / 1 (inspect).--existstrue/false; exit 0 on a hit, 2 on a miss — incl. resolve misses. Scriptable existence gate (find X --exists,inspect X --exists).--fields fqn,role,…--detail(surfacessignatureeven at brief); primarily a--format jsonlever.Why no
--brief?--detail briefalready provides identity-only output end-to-end (_BRIEF_NODE_KEYS+ the brief text tier). A separate--briefwould be a redundant alias that muddies the "detail is orthogonal to format" model. Verified and confirmed with the issue author.How
Two seams, no per-handler logic:
jrag_render.py—render()gainscount/exists/fieldskwargs (existstakes precedence overcount); newcount_results()/has_results()(shape-aware) are the single source of truth shared by--countoutput and the--existsexit code.jrag.py— a new_emit(env, args, …) → rcfunnel routes every command's success / not_found path (existing_render_listing/_emit_traversaldelegate to it). True usage/setup errors (missing index, kind guard,neighbors_v2failure) bypass it so the actionable message isn't hidden behind a count._core_parsercommands (status / microservices / vocab-index) are excluded: vocab-index doesn't render an envelope, so the flags would be silently ignored there (the repo's "inapplicable flags never silently ignored" rule).Tests
test_jrag_render.py): count/exists/fields shaping, shape-aware counting, precedence,--fieldsoverrides--detail, ignored on non-ok.test_jrag_locate.py):--existsexit code 0/2 on hit/miss (incl.inspectresolve-miss),--countmatches row count,--fieldsJSON projection.Docs re-advertised in
skills/explore-codebase-cli/SKILL.md+agents/explorer-rag-cli.mdand re-synced toinstall_data(sync_agent_artifacts.py --checkpasses).Closes #376.
🤖 Generated with Claude Code