v4.2.1 — batch_scrape backwards-compat + CLI batch fix
Highlights
- Retracts the v4.0.0 "breaking change" to
batch_scrapedefaults. A deep audit revealed it was a phantom at the MCP surface:server.js:544always defaultedformatsto['json'], but the internalBatchScrapeSchemadefaulted to['markdown']. Because params are validated at both layers and the MCP layer wins, MCP clients (Claude Code, Cursor, the CLI) were never broken. Only direct programmatic callers ofBatchScrapeTool.execute()ever saw the mismatch. v4.2.1 aligns the inner schema to['json']to close the latent gap. No migration needed for any caller — upgrades from 3.5.1 are clean. - Fixes the CLI
batchcommand —--format,--concurrency,--max-retriesflags were being silently dropped because the command passedoutput_format/concurrency/max_retriesto the tool, but the schema expectsformats/maxConcurrency/jobOptions.maxRetries. Zod's strip-mode silently dropped the unknown keys.
What changed
src/tools/advanced/batchScrape/schema.js— inner default['markdown']→['json'](matches MCP registration)src/cli/commands/batch.js— map CLI flags to actual schema keystests/unit/tools/advanced/batchScrape.test.js— 4 new regression/contract testsCHANGELOG.md— v4.0.0 entry retracted with postmortem; v4.2.1 entry addedIMPROVEMENT_ROADMAP_V4.md,PRD.md— annotated with the retraction
Install / upgrade
```bash
npm install -g crawlforge-mcp-server@4.2.1
```
Verification
- 11/11 batchScrape unit tests pass (10 existing + 1 new contract test)
- 3 new schema-default pin tests
- 6/6 CLI integration tests pass
- No regressions in 131-test D5.2 unit-test suite
Roadmap status
All phases of IMPROVEMENT_ROADMAP_V4.md complete: D1 ✓ D2 ✓ D3 ✓ D4 ✓ D5 ✓.