Official CrawlFox SDKs and adapters for https://api.crawlfox.io.
Get a key from the dashboard. Every request uses Authorization: Bearer. Do not ship keys in browsers.
| Package | Install | Description |
|---|---|---|
| crawlfox | npm install crawlfox |
JavaScript / TypeScript SDK |
| crawlfox-py | pip install crawlfox-py |
Python SDK (sync and async) |
| crawlfox (Go) | go get github.com/Automote-LLC/crawlfox-integrations/packages/go@v0.1.0 |
Go SDK |
| crawlfox (Rust) | crawlfox = "0.1.0" (crates.io) |
Rust SDK |
| crawlfox (PHP) | composer require crawlfox/crawlfox |
PHP SDK |
| crawlfox (Java) | io.crawlfox:crawlfox |
Java 17+ SDK |
| crawlfox (.NET) | Crawlfox NuGet |
.NET SDK |
| crawlfox (Ruby) | gem install crawlfox |
Ruby SDK |
| crawlfox (Elixir) | {:crawlfox, "~> 0.1.0"} |
Elixir SDK |
| crawlfox-aisdk | npm install crawlfox-aisdk |
Vercel AI SDK tools |
| langchain-crawlfox | pip install langchain-crawlfox |
LangChain loaders and tools |
| crawlfox-llamaindex | pip install crawlfox-llamaindex |
LlamaIndex web reader |
| crawlfox-crewai | pip install crawlfox-crewai |
CrewAI agent tools |
| n8n-nodes-crawlfox | Community node for n8n | Scrape and search in workflows |
| zapier | Zapier Platform CLI app | Scrape and search actions |
JavaScript on npm: https://www.npmjs.com/package/crawlfox
| Method | Path | What you get |
|---|---|---|
| scrape | POST /v1/scrape |
One URL. Formats: markdown, html, rawHtml, json, links, images, emails |
| scrape GET | GET /v1/scrape/:url |
Lightweight scrape of an encoded URL |
| batch | POST /v1/batch |
Many URLs, same formats, order preserved |
| search | POST /v1/search |
Google, Bing, or DuckDuckGo |
| search stream | POST /v1/search/stream |
NDJSON page / done / error |
| logs | GET /v1/logs/:id |
Status and timing (metadata.scrapeId or search id) |
| log result | GET /v1/logs/:id/result |
Stored scrape body |
| map | POST /v1/map |
Site URLs without scraping (links, optional limit) |
| crawl | POST /v1/crawl |
Start multi-page crawl; poll GET /v1/crawl/:id, cancel, errors |
Credits: 1 per scraped page, 1 per 10 requested search results, 1 per map call, 1 per crawled page per format.
Contract: contract/api.yml — every SDK must expose the methods listed there.
npm run check-contract # CI gate
npm run sdkctl:docs # scaffold drafts + snippets + metadata + verify
npm run sdkctl:docs -- --apply-app ../crawlfox-app
npm run sdkctl:publish:dry-run # no registry uploadCursor skill sdk-docs (.cursor/skills/sdk-docs/SKILL.md): trigger when a new API feature ships — AI rewrites per-language READMEs for your review before publish.
Auth: CRAWLFOX_API_KEY or constructor apiKey / api_key / APIKey.
Agents can use the same API through https://mcp.crawlfox.io/mcp (OAuth). That is not this repo. See https://docs.crawlfox.io
# JavaScript SDK
cd packages/js && npm install && npm test && npm run build
# Python SDK
cd packages/python && pip install -e ".[dev]" && pytest
# Go SDK
cd packages/go && go test ./...
# Rust SDK
cd packages/rust && cargo test
# PHP
cd packages/php && composer install && vendor/bin/phpunit
# Java
cd packages/java && mvn test
# .NET
cd packages/dotnet && dotnet test Crawlfox.Tests/Crawlfox.Tests.csproj
# Ruby
cd packages/ruby && ruby -Ilib:test test/client_test.rb
# Elixir
cd packages/elixir && mix deps.get && mix testSet CRAWLFOX_API_KEY (and optionally CRAWLFOX_API_URL) to run live scrape/search/batch tests against production.
GitHub Actions CI runs all nine language SDKs. Add repo secrets CRAWLFOX_API_KEY and optional CRAWLFOX_API_URL to enable live tests in CI.