Skip to content

v1.2.0 — fetch_url + API 2026-05-22 sync

Choose a tag to compare

View all tags
@kshvab kshvab released this 22 May 22:26
v1.2.0
c6d9c08

Description

Mirrors the @competlab/sdk@1.2.0 release. Brings the MCP surface in line with the 2026-05-22 API ship.

✨ New tool: fetch_url

Fetch any URL with automatic JS-rendering and common bot-protection handling. Returns body, headers, and cleanStats. Optional cleanHtml strips HTML noise while preserving text content — significant token-cost win for LLM consumption (typical: ~90% reduction on content-heavy pages).

  • Rate limit: 60 req/min per API key (tighter than the 1000/min default for other free tools)
  • Honest degradation: headersAvailable: false is surfaced when targets use advanced behavioral fingerprinting
  • Pass-through validation: out-of-range knobs, IPv4/IPv6/localhost hosts, and the no-output combo (bodyNeeded: false, headersNeeded: false) return precise invalid_parameters errors

📡 New degradation discriminators (pass-through)

The MCP server stays a thin pass-through over the API. The 2026-05-22 response-shape additions flow through automatically — no client changes needed unless you want to handle the degraded case explicitly:

  • signalsAvailable on get_trust_signals_scan and per-competitor security headers
  • partialDetection on get_tech_stack_scan (HTML-based detection still fires; only header-based hosting/CDN signals are affected)
  • securitySignalsAvailable + nullable securityGrade / securityScore on get_tech_trust_dashboard
  • details.reason: 'site_uses_behavioral_protection' on agent-adoption link-headers / cache-header-hygiene checks (scored: false, weight: 0 — no score impact)
  • New BEHAVIORAL_PROTECTION_ADOPTED alert type + previousSecurityProfile context field on list_alerts
  • Counter rename: meta.counters.n8nCalls → meta.counters.heavyFetchCalls on get_agent_adoption_scan

Tool descriptions for the 5 affected tools now include 1-line advisory notes so agents handle the degraded case cleanly without re-reading the spec.

📊 Counts

  • Tool count: 32 → 33
  • Free tools: 8 → 9
  • Read-only: 29 → 30

✅ Verified

Smoke-tested against prod:

  • competlab.com (healthy) → headersAvailable: true, full body + headers, cleanStats showing 92.95% reduction
  • g2.com (bot-protected) → headersAvailable: false, body populated, headers correctly absent
  • Negative test (bodyNeeded: false, headersNeeded: false) → returns the locked 400 invalid_parameters response verbatim

🔗 Companion release

Assets 2
Loading