Skip to content

v1.0.0

Latest
Latest

Choose a tag to compare

View all tags
@arjunken arjunken released this 24 Jul 06:01
· 5 commits to main since this release
v1.0.0
ed37573

CrawlDock v1.0.0

Web Search MCP Server for any MCP-compliant applications with Google, DuckDuckGo, and web scraping support.

🚀 What's New

CrawlDock is a comprehensive Model Context Protocol (MCP) server that provides web search capabilities for any MCP-compliant applications. Features Google Custom Search API integration with DuckDuckGo and web scraping fallbacks, optimized for LLM consumption.

✨ Features

  • 🔍 Multi-Engine Search: Google Custom Search API (primary), DuckDuckGo API, and web scraping fallbacks
  • 🤖 LLM-Optimized Results: Structured responses with relevance scoring and confidence metrics
  • 🔑 Flexible Authentication: Bring your own Google API key or use free alternatives
  • 🛡️ Rate Limiting: Built-in 500 requests/day limit for safe usage
  • 📊 Comprehensive Logging: Detailed logs with correlation IDs and LLM-specific events
  • ⚡ Fast Fallbacks: Automatic failover between search engines with retry logic
  • 🎯 Rich Results: Titles, URLs, snippets, metadata, and source attribution

🛠 Supported MCP Clients

  • LM Studio - Full integration with web search toggle
  • Claude Desktop - Native MCP support
  • Ollama - Plugin-based integration
  • Any MCP-compliant application - Universal compatibility

�� Installation

git clone https://github.com/yourusername/crawldock.git
cd crawldock
npm install
npm run build

📋 Configuration

Add to your MCP client configuration:

{
  "mcpServers": {
    "web-search": {
      "command": "node",
      "args": ["dist/index.js"],
      "cwd": "/path/to/crawldock"
    }
  }
}

🎯 Usage

Once configured, you can use CrawlDock in your conversations:

  • Basic Web Search: "Search for the latest AI developments"
  • Specific Queries: "What are the current trends in machine learning?"
  • Real-time Information: Get up-to-date information from the web

🔒 Security

  • API keys stored securely in environment variables
  • Rate limiting prevents abuse
  • Comprehensive error handling
  • No sensitive data in logs

�� Performance

  • Multi-engine fallback ensures reliability
  • Optimized for LLM consumption
  • Fast response times with caching
  • Detailed performance monitoring

Thank you for using CrawlDock! ��

Full Changelog: https://github.com/arjunken/crawldock/commits/v1.0.0

Assets 3
Loading