Every ocr review takes 15–30 min — how to reduce wall-clock time? (concurrency / tool-call budget?)

[Dream-wu]

Environment

• OCR version: v1.12.10
• Model: qwen3.8-max via an Anthropic-compatible endpoint (/v1/messages, use_anthropic=true)
• LLM gateway: a company-hosted proxy (may rate-limit / serialize requests)
• Invocation: ocr review --from origin/<target> --to origin/<source> --rule <rule.json> --format json
• Deployed as a service (Docker, Linux)

Problem

Every branch review takes 15–30 minutes. I want to bring this down for interactive/CI use.

Real data from one review (13 files, 24 comments, 0 severe, 30m27s)

Resource consumption:

Token type	Count
Input	4,837,617
Output	86,996
Cache read	4,502,435
Cache write	333,570
Total	4,924,613

Tool calls (193 total):

Tool	Calls
code_search	99
file_read	71
file_read_diff	10
file_find	7
code_comment	6

The files were split into 4 sub-agent bundles. There was also 1 auxiliary plan_task LLM request that timed out ("结果可能未经过完整的后置过滤").

My read: 1827s / 193 ≈ 9.5s per tool round-trip, so wall-time looks dominated by number of sequential LLM turns × per-turn latency, not by output size (only 87K output tokens).

Questions

1. Concurrency — the README says file bundling "naturally supports concurrent review". Are the sub-agent bundles run in parallel or serially by default? Is there a --max-concurrency / env var to raise parallelism across bundles? This looks like the biggest lever.
2. Tool-call budget — can I cap code_search/file_read rounds (or disable repo-level context lookup) to trade some depth for speed? A per-file "max agent steps / turns" setting?
3. Auxiliary modules — plan / positioning / reflection phases each add LLM passes. Can any be turned off (or made optional) for a "fast mode"?
4. Per-request timeout & retries — the plan_task timeout suggests slow auxiliary calls stall the run. Is there a configurable request timeout / retry count so one slow call doesn't dominate?
5. Gateway rate-limiting — if my proxy serializes requests server-side, OCR's concurrency won't help. Is there a way to see whether OCR is issuing requests concurrently (e.g. via the OpenTelemetry integration / session viewer)?
6. Rule file scope — I apply a **/* rule (incl. an output-language instruction) to every file. Does a broad rule increase tokens/turns, and is per-path rule matching cheaper?
7. Any documented benchmark tuning guide / recommended settings to minimize Avg Time (the README benchmark table) that I've missed?

What I'd like

A "fast preset" or a couple of flags/env vars to cut wall-clock to a few minutes for small diffs, accepting lower recall — similar to the precision/recall trade-off you already document.

Happy to provide sanitized logs / a session trace. Thanks!

[leenlbma]

Bundles do run concurrently — the “naturally supports concurrent review” line is literal. Defaults and the knobs that match your six questions:

1. Concurrency

ocr review defaults to --concurrency 8 (“max concurrent subtasks”). Each file bundle is one subtask. Raise it if the gateway allows parallel requests; lower it if the proxy serializes or rate-limits (the FAQ’s “CI run is much slower than local” section suggests e.g. --concurrency 4 under throttling).

2. Tool-call budget

--max-tools N caps tool-call rounds per subtask (0 = template default; values only take effect when greater than that default; minimum meaningful value is 50). That is the direct lever on code_search / file_read volume.

Also try --effort low (1 review round) vs the default medium (2) / high (3). Fewer rounds usually means fewer tool turns.

3. Auxiliary modules

--no-plan exists on ocr scan, not on ocr review. For review, use --effort low and optionally --no-filter (skips the LLM post-filter on comments). There is no separate “fast mode” flag for review beyond these.

4. Timeouts

Two independent budgets:

• --timeout 15 — concurrent task budget in minutes (LLM + retries; 0 = unlimited). Default 15.
• Per-request: OCR_LLM_TIMEOUT or the provider’s timeout_sec in seconds (default 300). Raise this if plan_task is timing out on a slow gateway.

5. Seeing concurrency

Use the OpenTelemetry integration and watch LLM metrics (ocr.llm.tokens_used, etc. — see the Telemetry docs). If spans for different bundles overlap in time, requests are concurrent; if they never overlap, the gateway or --concurrency 1 is serializing them.

6. Rule scope

A **/* rule is applied to every selected file, so its instructions are in every prompt. Prefer narrower path rules for language/style, and keep a small shared rule for output language only — that usually cuts tokens and turns.

Practical starting point for your 30m case

ocr review --from origin/<target> --to origin/<source> \
  --rule <rule.json> --format json \
  --effort low \
  --concurrency 8 \
  --max-tools 50 \
  --timeout 20

If wall time barely moves when you raise --concurrency, the company proxy is almost certainly serializing; fix that before tuning OCR further.