research-mcp

One MCP server + OpenAI plugin that searches 9 sources, scrapes full content, reranks, and synthesizes a cited answer with a free LLM.

Sources (9)

Source	Default?	Auth	Notes
Firecrawl (your fork)	✅	FIRECRAWL_API_URL + optional FIRECRAWL_API_KEY	Calls POST /v1/search
GitHub Trending	✅	none	HTML scrape, no key
arXiv	✅	none	Official arxiv lib
Tavily	✅	TAVILY_API_KEY	Primary fallback scraper, 1k/mo free
OSS Insight	✅	none	Trending repos, free API
dev.to	✅	none	Public API
libhunt	❌ opt-in	none	HTML scrape
X.com (Twitter)	❌ opt-in	TWITTERAPI_IO_KEY	No free public X API
Google SERP	❌ opt-in	SERPER_API_KEY or SERPAPI_API_KEY	No free public SERP API

Free LLM & reranker

• Summarizer: Groq (llama-3.3-70b-versatile, free) → Gemini 2.0 Flash → OpenRouter free → extractive fallback
• Reranker: Cohere rerank-english-v3.0 (1k/mo free) → Jina v2 → local term-overlap
• Scraper: Firecrawl → Tavily extract → Jina Reader (r.jina.ai, no key)

Install

cd research-mcp
pip install -e .
cp .env.example .env
# fill in whichever keys you have; the rest are optional

Run the MCP (stdio, for Claude Desktop / goose)

python -m mcp_server

Add to your Claude Desktop config:

{
  "mcpServers": {
    "research": {
      "command": "python",
      "args": ["-m", "mcp_server"],
      "cwd": "C:\\Users\\kevin\\research-mcp"
    }
  }
}

Run the OpenAI plugin shim (HTTPS)

python -m openai_plugin.app
# then serve over HTTPS via Caddy/Nginx and point ChatGPT at
# https://your.host/.well-known/openai-plugin.json

Tools exposed

Tool	Purpose
research(query, sources?, max_results?, style?)	End-to-end: search → scrape → rerank → summarize
search_all(query, sources?, max_results?)	Search only, no LLM
scrape_url(url, method?)	Single URL scrape
rerank_docs(query, docs, model?)	Rerank a list of strings
summarize_docs(docs, query, style?, model?)	Final synthesis

Smoke test (no keys required)

python tests/smoke.py

Runs arxiv → local rerank → extractive summary. Proves the pipeline is wired correctly without spending any free-tier quota.

Docker

docker compose --profile mcp up --build       # MCP over stdio
docker compose --profile plugin up --build    # OpenAI plugin on :8000

Layout

mcp_server/
  server.py                # fastmcp entry
  config.py                # env-driven settings
  adapters/                # one file per source, normalized SearchResult
  tools/
    search.py              # fan-out + dedupe
    scrape.py              # Firecrawl / Tavily / Jina
    rerank.py              # Cohere / Jina / local
    summarize.py           # Groq / Gemini / OpenRouter
openai_plugin/
  app.py                   # FastAPI shim re-exporting the same tools
  manifest.json
tests/smoke.py
Dockerfile, docker-compose.yml