Architecture

Architecture

web-search-mcp follows a modular architecture with a clear separation of concerns to ensure maintainability and extensibility.

Project Structure

web_search_mcp/
├── server.py              # FastMCP entry point, registers all 16 tools
├── __init__.py            # Package metadata (version 0.2.0)
├── _config/               # Application settings & limits
│   ├── settings.py        #   pydantic-settings: API keys, env vars
│   └── limits.py          #   Rate limit constants
├── _http/                 # Shared HTTP infrastructure
│   └── client.py          #   httpx-based client with timeouts
├── _models/               # Pydantic schemas
│   ├── requests.py        #   Input validation models
│   ├── responses.py       #   Output response models
│   └── types.py           #   Shared type aliases
├── _utils/                # Shared utilities
│   ├── formatting.py      #   Error formatting, markdown helpers
│   ├── rate_limiter.py    #   Token-bucket rate limiting
│   └── scoring.py         #   Relevance scoring for multi-source results
├── search/                # General web search engines
│   ├── ddg.py             #   DuckDuckGo: search + content extraction
│   └── exa.py             #   Exa AI: semantic web search
├── social/                # Social & community platforms
│   ├── github.py          #   GitHub Issues/PRs search
│   ├── hackernews.py      #   Hacker News via Algolia API
│   ├── reddit/            #   Keyless RSS → shreddit enrichment
│   │   ├── client.py      #     RSS & shreddit HTTP layer
│   │   ├── engine.py      #     Search orchestration
│   │   ├── models.py      #     Data models
│   │   └── parsers.py     #     XML/JSON parsing
│   └── x.py               #   X/Twitter via vendored Bird CLI
└── tools/                 # Research & developer utility tools
    ├── arxiv.py           #   arXiv academic paper search
    ├── compare.py         #   Side-by-side technology comparison
    ├── errors.py          #   Error message analysis + SO search
    ├── groq_client.py     #   Shared Groq API client with retries
    ├── groq_tools.py      #   Groq-powered search & page analysis
    ├── registries.py      #   Package registry lookup (npm, PyPI, etc.)
    └── wikipedia.py       #   Wikipedia search & article reading

Design Principles

1. Modular Tooling: Each search engine or extraction method is isolated in its own module.
2. Strict Validation: Pydantic models ensure that all inputs and outputs adhere to the expected schema.
3. Consistent Error Handling: _utils/formatting.py provides centralized error formatting so all MCP tools return errors in a consistent shape.
4. Keyless by Default: Reddit, Hacker News, Wikipedia, arXiv, DuckDuckGo, and developer tools require no API keys. Only Groq tools and X/Twitter need credentials.
5. Tiered Enrichment: Sources like Reddit and GitHub use multi-tier pipelines (quick → default → deep) to balance speed against result depth and comment enrichment.
6. Plugin Ready: Ships with .claude-plugin/ manifest for installation as a Claude Code Plugin, distributing both MCP tools and reusable skills.

---

Home | Tools | Development | Configuration