MCP extract tools: outputSchema mismatches server response — crw_extract and crw_check_extract_status always fail client validation

[Zorgonatis]

Note: This bug report was generated by an AI agent (Sisyphus, running via OpenCode) as part of diagnosing a broken MCP integration. The analysis is based on reading the crw v0.25.2 source code and testing against a live self-hosted instance. All technical claims below are verifiable against the source.

Bug

Both extract-related MCP tools declare outputSchema values that don't match what the server actually returns, causing every extract call to fail MCP client-side schema validation.

Error

MCP error -32602: Structured content does not match the tool's output schema: data must have required property 'success'

Root Cause

crw_extract (start) — schema in crates/crw-mcp-proto/src/lib.rs:91-103 (extract_accepted_output_schema):

{
  "required": ["success", "id", "status", "urls"],
  "properties": {
    "success": {"type": "boolean"},
    "id": {"type": "string"},
    "status": {"type": "string", "enum": ["processing"]},
    "urls": {"type": "integer", "minimum": 0}
  },
  "additionalProperties": false
}

Server response (V2ExtractStartResponse in crates/crw-server/src/routes/v2/extract.rs:36-42):

{
  "success": true,
  "id": "...",
  "urlTrace": [],
  "warnings": ["..."],
  "replacement": "/v2/scrape"
}

Missing from response: status, urls (both required by schema)
Extra in response: urlTrace, warnings, replacement (all rejected by additionalProperties: false)

---

crw_check_extract_status — schema in crates/crw-mcp-proto/src/lib.rs:105-144 (extract_status_output_schema):

{
  "required": ["success", "id", "status", "results", "expiresAt", "creditsUsed", "tokensUsed"],
  "properties": {
    "success": {"type": "boolean"},
    "id": {"type": "string"},
    "status": {"type": "string", "enum": ["processing", "cancelling", "completed", "failed", "cancelled"]},
    "results": { ... },
    ...
  }
}

Server response (V2ExtractStatusResponse in crates/crw-server/src/routes/v2/extract.rs:100-110):

{
  "success": true,
  "status": "completed",
  "data": { ... },
  "error": null,
  "expiresAt": "...",
  "creditsUsed": 1,
  "tokensUsed": 488
}

Missing from response: id (not in the struct), results (has data instead)
Extra in response: data (not in schema)

Impact

Both extract tools are unusable via MCP — every call fails schema validation, even though the server processes the request correctly. The extraction feature itself works (verified via REST /v1/scrape with formats:["json"] + jsonSchema returning structured JSON from a glm-4.5-air LLM backend).

Version

crw v0.25.2 (server + MCP, same version)

Suggested Fix

Either:

1. Update the schemas to match the actual response shapes (add urlTrace/warnings/replacement to start schema, change results → data in status schema, remove id from status required), or
2. Update the server responses to match the schemas (add status/urls to start response, rename data → results in status response, add id echo back)

Option 1 is likely safer — the response shapes are already deployed and being consumed by REST clients.

[us]

hey @Zorgonatis thanks for reporting us! will fix it quickly!

[us]

thanks for the incredibly precise writeup, and for actually running it against a live instance. the failure was real and reproduced exactly as you described.

one nuance on the root cause: the mcp proxies call /v1/extract, not the deprecated /v2/extract route you were reading, so the declared outputSchemas were actually correct. the real problem was narrower: the /v1 extract start/status bodies were the only /v1 responses missing the success envelope that every other endpoint returns, and the proxies forward that body verbatim as structuredContent, so strict clients rejected it with the exact must have required property 'success' you hit.

the fix (#323) adds success back to the two /v1/extract responses (start = true, status = true unless the whole job failed), which lines extract up with scrape/search/map/crawl and fixes both proxy transports with no schema changes. added a regression test that validates the raw rest body against the advertised schema across every job status so this can't drift again.

it's merged to main now; since engine docker images build on release, it'll be in the next tagged release. thanks again for the great report.