perf(es): warn when --json is used with unbounded --max-docs in scroll-search

[ssh-esh]

Closes #211

When --json is active, scroll-search buffers all documents in memory. Since --max-docs defaults to unlimited, this can OOM on large indices. The NDJSON streaming path (default, no --json) is unaffected since it writes each doc to stdout immediately.

Options considered

• Option 1: Cap --max-docs
  • Approach: default to 10,000 in --json mode
  • Downside: silent data truncation — user gets incomplete results that look correct
• Option 2: Warn on stderr
  • Approach: emit a warning when --json + unbounded --max-docs
  • Downside: user could still OOM if they ignore the warning
• Option 3: Both
  • Approach: warn + cap, allow explicit override
  • Downside: same truncation risk as option 1; two behaviors for one flag depending on --json

Decision: option 2 (warn only)

I went with a stderr warning because silent truncation is worse than a loud failure. An OOM is at least obvious, while truncated results look correct but aren't. This is especially important for agents, which parse stdout and would silently act on incomplete data.

This aligns with how other CLIs handle unbounded JSON output:

• AWS CLI pagination provides --max-items but doesn't cap by default

Recent writing on agent-focused CLI design also recommends NDJSON for streaming (which we already do) and explicit user-set limits over silent caps:

• Agentic CLI Design: 7 Principles
• Designing CLI Tools for AI Agents

[github-actions]

✅MegaLinter analysis: Success

Descriptor	Linter	Files	Fixed	Errors	Warnings	Elapsed time
✅ COPYPASTE	jscpd	yes		no	no	9.12s
✅ REPOSITORY	gitleaks	yes		no	no	94.75s
✅ REPOSITORY	git_diff	yes		no	no	0.42s
✅ REPOSITORY	secretlint	yes		no	no	12.56s
✅ REPOSITORY	trivy	yes		no	no	17.67s
✅ TYPESCRIPT	eslint	2		0	0	3.86s

See detailed reports in MegaLinter artifacts
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository