feat: Lightweight agent follow-up — 1 capped tool-call for cross-file context

[ajianaz]

Problem

Cora v0.3 (context chain + bundling + rules) dramatically improves review quality through deterministic analysis. However, there are cases where the LLM identifies a potential issue but cannot confirm it without seeing additional code — e.g.:

• "This function call might return an error that's not handled, but I can't see the function signature"
• "This looks like it could be a race condition, but I need to see if the struct uses Mutex/RwLock"
• "The error type here seems overly broad, but I'd need to check how it's consumed elsewhere"

In v0.3, the LLM has NO way to request additional context. It must either guess (risking false positives) or skip (risking false negatives).

Why Not Full Agent (OCR-style)?

Alibaba's Open Code Review gives the agent unlimited tool calls (read_file, search_codebase, list_changed_files). This is powerful but:

Aspect	Full Agent (OCR)	Proposed: 1 Follow-Up
Max tool calls	Unlimited	1 (hard cap)
Token budget	20,000-40,000	+2,000 (predictable)
Predictability	Low	High
Cost control	Difficult	Built-in cap
Use case	Enterprise, deep audit	CLI pre-commit, CI

For a CLI tool used in pre-commit hooks and CI, cost predictability > maximum depth. One follow-up call is enough to resolve most "I need more context" situations.

Proposed Solution: Single Follow-Up Tool Call

Architecture

Phase 1: Normal Review (same as v0.3)
────────────────────────────────────────
  Diff + context chain + rule findings → LLM → issues JSON
  
  NEW: LLM can optionally output a "context_request" field:

Phase 2: Context Request (conditional, 1 call max)
───────────────────────────────────────────────────
  If LLM outputs context_request:
  {
    "issues": [...normal issues...],
    "context_request": {
      "file": "src/auth/validate.rs",
      "line_start": 22,
      "line_end": 45,
      "reason": "Need to verify validate_token returns Result, not Option"
    }
  }

  Deterministic handler:
  1. Read requested file + line range (max 2000 tokens)
  2. Inject into SECOND LLM call:
     "Additional context requested by reviewer:
      --- src/auth/validate.rs:22-45 ---
      fn validate_token(token: &str) -> Result<Claims, AuthError> { ... }
      
      Based on this additional context, re-review your findings.
      Keep confirmed issues, remove false positives, add new issues if any."

Phase 3: Merge Results
─────────────────────
  Final issues = original issues (re-validated) + new issues from follow-up
  If LLM dismisses a finding → removed from output
  If LLM adds new finding → added to output

Key Constraints

1. Exactly 1 follow-up call maximum — no chaining, no recursion
2. Context capped at 2,000 tokens — hard limit on file content read
3. Line range required — LLM must specify exact line range, not "whole file"
4. File must be in the context chain or changed files — prevents reading arbitrary files
5. Optional — LLM can skip this entirely (no context_request → no second call)

Token Cost Analysis

Scenario	v0.3 (no follow-up)	v0.4 (with follow-up)	Delta
Simple review (no context needed)	~11,500 tokens	~11,500 tokens	0%
Review needing context	~11,500 tokens (missed)	~13,500 tokens	+17%
Average across all PRs	~11,500 tokens	~12,000 tokens	+4%

Key insight: The follow-up is ONLY triggered when the LLM explicitly needs more context. For most reviews (~70%), no follow-up occurs (zero extra cost). For reviews that need it, the +2,000 token investment resolves ambiguity that would otherwise produce false positives/negatives.

Integration with v0.3 Features

Feature	Interaction
Context Chain (#114)	Follow-up is a FALLBACK — if context chain didn't resolve the needed symbol, the LLM can request it directly
File Bundling (#115)	Follow-up scoped to bundle — can only request files from the same bundle's changed files
Rule Engine (#116)	Rule findings are deterministic (no follow-up needed). Follow-up is only for LLM-identified issues
--progress	New event type: follow_up_requested, follow_up_complete

Prompt Engineering

Updated system prompt addition:

CONTEXT REQUEST (optional):
If you need to see additional code to verify a finding, add a "context_request" field:
{
  "context_request": {
    "file": "path/to/file.rs",
    "line_start": 10,
    "line_end": 30,
    "reason": "Brief explanation of what you need to verify"
  }
}
You will receive the requested code and a chance to update your findings.
Use this ONLY when you're uncertain about a finding — not for general exploration.
Most reviews should NOT need a context request.

Configuration

# .cora.yaml
review:
  agent:
    enabled: true              # default: true (follow-up available)
    max_follow_up_tokens: 2000 # context budget for follow-up (default: 2000)
    allow_follow_up: true      # can disable entirely for maximum speed

CLI Flags

cora review --base origin/develop            # follow-up enabled (default)
cora review --base origin/develop --no-agent  # disable follow-up entirely

--progress Events (New)

{"type":"follow_up_requested","file":"src/auth/validate.rs","lines":"22-45","reason":"Verify return type"}
{"type":"follow_up_complete","issues_confirmed":2,"issues_removed":1,"issues_added":1,"extra_tokens":1800}

Acceptance Criteria

• LLM can output optional context_request in review response
• Context request triggers exactly 1 file read + 1 LLM follow-up call
• Follow-up context capped at max_follow_up_tokens (hard limit)
• File in context_request must be from changed files or context chain
• Line range required — no "read whole file" requests
• Findings merged: confirmed kept, dismissed removed, new added
• --progress emits follow-up events
• review.agent.allow_follow_up: false disables feature
• --no-agent flag disables feature
• 70%+ of reviews produce NO follow-up (verify with benchmark)

Risks

Risk	Mitigation
LLM always requests context (cost increase)	Prompt guidance: "Most reviews should NOT need a context request"
LLM requests irrelevant files	File scope restricted to changed files + context chain
LLM changes its mind incorrectly	Final merge = original + follow-up. Original issues preserved unless explicitly dismissed
Additional latency (~5-10s for follow-up)	--no-agent flag for speed-critical CI; follow-up runs in parallel with other bundles

Future Expansion (v0.5+)

If demand exists, the architecture supports:

• max_follow_up_calls: 2 — allow 2 follow-ups for deep audit mode
• Per-bundle agent mode — agent enabled only for bundles flagged as "complex"
• search_codebase tool — limited grep within project (not full file read)
• Cost estimation — "This review will use approximately X tokens. Continue? [y/n]"

References

• Alibaba Open Code Review: "Scenario-tuned toolset" — agent reads files, searches codebase
• Cursor: Ctrl+K context fetch — user-requested, not agent-initiated
• Aider: "repository map" — pre-computed symbol map for agent context