Open-source research engine that extracts structured knowledge from any topic.
Feed it a question. Get back scored claims, entity graphs, and a synthesized report β not paragraphs of slop.
Quick Start β’ Use with AI Assistants β’ Drone Arsenal β’ Benchmarks β’ Architecture β’ API β’ Self-Hosting β’ Integrations
You ask Claude to research a topic. It gives you a confident-sounding summary based on training data that's months old. No sources. No structure. No way to verify.
You ask Perplexity. Better β it cites sources. But the output is still unstructured text. You can't query it, cross-reference it, or build on it.
BeHive is different. It produces machine-readable intelligence: typed claims with confidence scores, entity relationship graphs, and structured JSON you can pipe into any downstream system.
Your AI assistant β BeHive β Verified, structured, scored knowledge
βββ 363 claims (avg quality 0.77)
βββ 42 entities with relationships
βββ Synthesized report with citations
pip install behive
# Set your LLM API key (you use YOUR OWN subscription β BeHive costs nothing)
export ANTHROPIC_API_KEY=your-key # or OPENAI_API_KEY, or AWS creds for Bedrock
# PostgreSQL required for storage (or use Docker below)
export BEHIVE_DB_URL=postgresql://user:***@localhost:5432/behive
# Start the server
behive serveFastest path β Docker Compose (PostgreSQL included):
git clone https://github.com/qa10devteam/behive && cd behive
echo "ANTHROPIC_API_KEY=your-key" > .env
docker compose up -d
# API at http://localhost:8091Full install (stealth drones, content extraction, NLP processing):
pip install "behive[all]"
β οΈ GPU/CUDA note:behive[all]does NOT include GPU dependencies. If you need vector embeddings (Qdrant), install separately:pip install "behive[qdrant]"β this pulls PyTorch + sentence-transformers (~4GB with CUDA). For CPU-only machines, install torch CPU-only first:pip install torch --index-url https://download.pytorch.org/whl/cpu
Or pick what you need:
pip install "behive[stealth]" # curl_cffi, primp, nodriver, patchright
pip install "behive[harvest]" # trafilatura, newspaper4k, PyMuPDF, crawl4ai
pip install "behive[process]" # rapidfuzz, spacy, litellm, tiktoken
pip install "behive[mcp,api]" # MCP server + REST APIThat's it. BeHive is now running:
- API β
http://localhost:8091(REST endpoints) - MCP β
http://localhost:8090/mcp(for AI assistants) - Docs β
http://localhost:8091/docs(Swagger UI)
BeHive doesn't force you into one model. You choose what runs each pipeline stage:
| Stage | Role | Recommended |
|---|---|---|
| scout | Query generation, source discovery | Haiku / GPT-4o-mini / local |
| harvest | Relevance filtering, content triage | Haiku / GPT-4o-mini / local |
| process | Claim extraction, quality scoring | Haiku or Sonnet |
| synth | Report synthesis, deduplication | Sonnet / Opus / GPT-4o |
# Apply a preset
behive config --preset balanced # Haiku collects, Sonnet synthesizes (~$1.50/mission)
behive config --preset budget # Haiku everywhere (~$0.30/mission)
behive config --preset quality # Sonnet everywhere (~$4.00/mission)
behive config --preset local # Your own LLM server ($0.00/mission)behive config --quick # Pick one model for everything
behive config --full # Choose model per stage (interactive)# Set a single stage
behive config --stage synth --model claude-opus
behive config --stage scout --model ollama/deepseek-r1
# Check current config
behive config --showexport BEHIVE_MODEL_SCOUT=ollama/llama3.1
export BEHIVE_MODEL_SYNTH=anthropic/claude-sonnet-4-20250514
behive servePriority: BEHIVE_MODEL_{STAGE} > BEHIVE_MODEL > config.yaml > defaults
| Preset | Model String |
|---|---|
claude-haiku |
anthropic/claude-haiku-4-5-20251001 |
claude-sonnet |
anthropic/claude-sonnet-4-20250514 |
claude-opus |
anthropic/claude-opus-4-20250514 |
gpt-4o-mini |
openai/gpt-4o-mini |
gpt-4o |
openai/gpt-4o |
gpt-4.1 |
openai/gpt-4.1 |
gemini-flash |
google/gemini-2.5-flash |
gemini-pro |
google/gemini-2.5-pro |
bedrock-haiku |
bedrock/us.anthropic.claude-haiku-4-5-... |
bedrock-sonnet |
bedrock/us.anthropic.claude-sonnet-4-6-... |
local |
openai/local-model (any OpenAI-compatible server) |
ollama |
ollama/llama3.1 |
Or pass any litellm-compatible model string directly.
You bring your Claude subscription. BeHive adds research superpowers. No extra cost from us.
Step 1: Install and start BeHive:
pip install behive
export ANTHROPIC_API_KEY=*** # your own key
behive serveStep 2: Open Claude Desktop β Settings β Developer β Edit Config β paste:
{
"mcpServers": {
"behive": {
"url": "http://localhost:8090/mcp",
"transport": "streamable-http"
}
}
}Step 3: Restart Claude Desktop. Done. Now ask:
"Research the EU AI Act enforcement timeline and penalties"
Claude will call BeHive automatically, search across multiple sources, and return scored claims instead of guessing from training data.
What happens under the hood
You ask Claude a question
β
Claude calls BeHive MCP tool "research_topic"
β
BeHive scouts 64 APIs (38 free, 26 BYOK), fetches 1000+ URLs via stealth drones
β
Your LLM key extracts claims (Claude Haiku = ~$0.50 per mission)
β
BeHive scores, deduplicates, builds knowledge graph
β
Returns structured report to Claude
β
Claude presents findings with confidence scores and source links
Cost: ~$0.30β$2.00 per research mission (your Anthropic/OpenAI tokens). BeHive itself: free forever (MIT license).
Step 1: Start BeHive on a server with a public URL (or use tunneling):
pip install behive
export OPENAI_API_KEY=*** # your own key
behive serve --host 0.0.0.0
# Expose with a tunnel (for testing):
# npx cloudflared tunnel --url http://localhost:8091Step 2: Create a Custom GPT at chat.openai.com/gpts/editor:
- Name: "Deep Researcher (BeHive)"
- Instructions: "You are a research analyst. Use the BeHive actions to research topics. Always cite claim confidence scores."
- Actions β Import URL: paste your server URL +
/openapi.json
Or manually add this schema:
openapi: 3.1.0
info:
title: BeHive Research API
version: 0.3.0
servers:
- url: https://*** paths:
/research:
post:
operationId: startResearch
summary: Start a deep research mission
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [query]
properties:
query:
type: string
description: Research topic or question
depth:
type: integer
default: 3
description: 1=quick, 3=standard, 5=deep
responses:
'200':
description: Mission started successfully
/research/{mission_id}:
get:
operationId: getResearchResults
summary: Get completed research with scored claims
parameters:
- name: mission_id
in: path
required: true
schema:
type: string
responses:
'200':
description: Research results with claims and report
/claims/search:
get:
operationId: searchKnowledge
summary: Search across all previously researched knowledge
parameters:
- name: q
in: query
required: true
schema:
type: string
- name: limit
in: query
schema:
type: integer
default: 20
responses:
'200':
description: Matching claims with scoresStep 3: Use your Custom GPT. Ask: "Research quantum computing breakthroughs 2026"
Any editor or tool supporting MCP works identically to Claude Desktop:
{
"mcpServers": {
"behive": {
"url": "http://localhost:8090/mcp",
"transport": "streamable-http"
}
}
}Available MCP tools:
| Tool | Description |
|---|---|
research_topic |
Start a deep research mission (returns job_id) |
mission_status |
Poll running mission progress |
get_report |
Get synthesized report for completed mission |
search_knowledge |
Search all previously extracted claims |
Hermes Agent (automatic β skill already published):
# BeHive skill auto-loads when you ask Hermes to research anything
# Just ensure behive serve is running on the same machine
behive serveOpenClaw:
# Install from integrations directory
cp integrations/openclaw/SKILL.md ~/.openclaw/skills/behive-research.mdBeHive doesn't just search the web. It deploys stealth drones β multi-layered fetch agents that break through anti-bot defenses, paywalls, and rate limits.
Every URL goes through an escalation cascade. If Layer 1 gets blocked, Layer 2 fires. All the way to Layer 8.
Layer 1 β DIRECT β aiohttp + full Chrome 131 headers
Layer 2 β UA ROTATION β 10 browser fingerprints (Chrome/Firefox/Safari/Edge)
Layer 3 β curl_cffi β TLS impersonation (JA3/JA4 fingerprint matching)
Layer 4 β primp β Rust-native TLS, newer fingerprints than curl_cffi
Layer 5 β nodriver β Headless Chrome via CDP, passes Cloudflare Bot Management
Layer 6 β patchright β Stealth Playwright (no Runtime.enable/Console.enable leak)
Layer 7 β Jina relay β r.jina.ai proxy (paywall + captcha bypass)
Layer 8 β Archives β Wayback Machine + archive.org fallback
| Defense | How |
|---|---|
| Cloudflare | Detected β escalate to nodriver/patchright (JS challenge solved) |
| DataDome | TLS fingerprint rotation (primp/curl_cffi) |
| Akamai Bot Manager | CDP-based headless + real browser UA pool |
| Rate limits | Automatic backoff + UA rotation + parallel diversification |
| Paywalls | Jina relay proxy + archive.org cache |
| Turnstile CAPTCHA | patchright stealth Playwright |
| 403/429 blocks | Smart retry with escalation, never hammer the same layer |
ββββ HEAD sweep (974+ URLs, async semaphore) ββββ
β β
βΌ βΌ
βββββββββββββββββββ ββββββββββββββββββ
β Resource Router β β Domain Recon β
β (8 resource β β (tier scoring β
β types detected)β β reputation) β
ββββββββββ¬βββββββββ βββββββββ¬βββββββββ
β β
ββββββββββββΌβββββββββββ¬βββββββββββ β
βΌ βΌ βΌ βΌ βΌ
api_bee pdf_drone std_drone heavy_drone domain_score
(70 APIs) (VLM parse) (Layer 1-8) (patchright) (0.0 - 1.0)
Routing decisions per resource type:
api_endpointβ Direct API bee (structured JSON, no parsing needed)pdfβ PDF drone (Vision LLM extraction)static_htmlβ Standard drone (Layer 1-4 usually sufficient)spaβ Heavy drone (Layer 5-6, needs JS execution)paywallβ Jina relay or archive fallbackrss_feedβ RSS bee (structured, fast)database_portalβ Dedicated connector (custom scraping logic)
Scout bees don't just Google. They query specialized APIs across 37 categories:
| Category | APIs | Examples |
|---|---|---|
| Academic | 5 | arXiv, Semantic Scholar, CrossRef, OpenAlex, CORE |
| Financial | 6 | SEC EDGAR, Yahoo Finance, FRED, ECB, World Bank |
| Government | 5 | TED (EU procurement), SAM.gov, UK FTS, BZP (Poland), GUS |
| Security | 6 | CVE/NVD, Shodan, VirusTotal, AbuseIPDB |
| Development | 8 | GitHub, npm, PyPI, crates.io, Docker Hub, Homebrew |
| ML/AI | 5 | HuggingFace, Papers With Code, Replicate, Ollama |
| News | 4 | NewsAPI, GNews, TheNewsAPI, Mediastack |
| Crypto | 2 | CoinGecko, CoinMarketCap |
| Patents | 1 | Google Patents (via SerpAPI) |
| Medical | 1 | PubMed/NCBI |
| ... | 25+ | Trade, geopolitics, environment, demographics, ... |
Total: 70 APIs, 125 endpoints β each checked per-mission based on topic relevance.
Real results. No cherry-picking. Scale 30 (standard depth).
Hardware: EC2 g6.24xlarge β 4Γ NVIDIA L4 (92 GB VRAM), 96 vCPU, 384 GB RAM
Models: Bedrock Claude Haiku (bulk extraction) + Sonnet (enrichment), SGLang/Qwen on local GPUs
| Topic | Claims | Avg Quality | Duration | Sources |
|---|---|---|---|---|
| NVIDIA GPU market 2026 | 290 | 0.797 | 8 min | 234 |
| OpenAI GPT-5 capabilities | 574 | 0.789 | 12 min | 174 |
| EU AI Act enforcement | 267 | 0.759 | 6 min | 130 |
| Perplexity AI business model | 267 | 0.759 | 7 min | 150 |
| Meta Llama 4 architecture | 568 | 0.821 | 11 min | 198 |
Quality score meaning:
0.90+β Exceptional: specific numbers, dates, sources, fully verifiable0.82+β Excellent: multi-source corroboration, publication-ready (top 25% of missions)0.75+β Good: useful intelligence with some specifics0.65+β Acceptable: general facts, entered into DB<0.55β Rejected: too vague, not stored
Honest scoring, no tricks. No sigmoid rescaling, no artificial inflation. The score is a weighted average of specificity, information density, uniqueness, verifiability, and structure.
ββββββββββββββββββββββββββββββββββββ
β BeHive Pipeline β
ββββββββββββββββββββββββββββββββββββ
β
βββββββββββββ¬ββββββββββββ¬ββββββββ΄ββββββββ¬ββββββββββββ¬ββββββββββββ
βΌ βΌ βΌ βΌ βΌ βΌ
βββββββββββ βββββββββββ ββββββββββββ ββββββββββββ βββββββββββ ββββββββββ
β SCOUT β β HARVEST β β PROCESS β β V4 β β SYNTH β β GRAPH β
β β β β β β β β β β β β
β Queen β β Parallelβ β BeeHive β β Haiku β β Claude β β Neo4j β
β plans β β HTTP β β fast β β extract β β report β β entity β
β 5 axes β β 1000+ β β extract β β + Sonnet β β + cite β β fuse β
β Γ N β β URLs β β + score β β enrich β β β β β
βββββββββββ βββββββββββ ββββββββββββ ββββββββββββ βββββββββββ ββββββββββ
β β β β β β
β β βΌ β β β
β β ββββββββββββββββ β β β
β β β Quality Gate β β β β
β β β conf β₯ 0.55 β β β β
β β β dedup 0.60 β β β β
β β ββββββββββββββββ β β β
β β β β β β
ββββββββββββββββ΄βββββββββββββ΄βββββββββββββ΄βββββββββββββ΄βββββββββββ
β
βββββββββββ΄ββββββββββ
β PostgreSQL β
β Claims + KG β
β 25K+ records β
βββββββββββββββββββββ
What makes it different from GPT-Researcher:
- Dual-model extraction β Fast model (Haiku) for bulk extraction, powerful model (Sonnet) for enriching thin claims. Not just "summarize this page."
- Quality scoring β Every claim gets a 0.0-1.0 score. Below threshold = rejected. No filler.
- Knowledge graph β Entities and relationships persist across missions. Research compounds.
- 64 API sources β Not just web search. SEC filings, arXiv, patent databases, government APIs.
- Deduplication β Jaccard 0.60 threshold prevents the same fact from different sources inflating counts.
BeHive exposes a REST API (port 8091) and MCP server (port 8090).
curl -X POST http://localhost:8091/research \
-H "Content-Type: application/json" \
-d '{
"query": "SpaceX Starship launch cadence 2026",
"depth": 3,
"scale": 30
}'
# β {"job_id": "hive_1785227949_815112", "status": "started"}curl -N http://localhost:8091/research/hive_1785227949_815112/eventsevent: start
data: {"topic": "SpaceX Starship...", "status": "scout"}
event: phase
data: {"phase": "process", "event": "started"}
event: claims
data: {"count": 142, "avg_quality": 0.77, "above_082": 23, "new_since_last": 18}
event: done
data: {"total_claims": 363, "avg_quality": 0.77, "sources": 64}
curl http://localhost:8091/research/hive_1785227949_815112/report
# β {"synthesis": "## SpaceX Starship...", "claims_count": 363, ...}# Full-text search across all missions
curl "http://localhost:8091/search?query=NVIDIA+revenue&limit=20"
# Entity intelligence
curl http://localhost:8091/intelligence/entity/NVIDIA
# Network graph (2-hop neighborhood)
curl "http://localhost:8091/intelligence/network/OpenAI?depth=2"| Method | Path | Description |
|---|---|---|
POST |
/research |
Start new mission |
GET |
/research/{id}/status |
Check progress |
GET |
/research/{id}/events |
SSE stream |
GET |
/research/{id}/report |
Get synthesis |
GET |
/search |
Query claims |
GET |
/intelligence/entity/{name} |
Entity details |
GET |
/intelligence/network/{name} |
Relationship graph |
GET |
/intelligence/stats |
System statistics |
Full Swagger docs: http://localhost:8091/docs
BeHive implements the Model Context Protocol β the emerging standard for AI tool connectivity.
{
"mcpServers": {
"behive": {
"url": "http://localhost:8090/mcp",
"transport": "streamable-http"
}
}
}Compatible with:
- Claude Desktop / Claude Code
- Cursor IDE
- Windsurf
- n8n (via MCP node)
- Any MCP-compatible client
Tools exposed:
| Tool | Description |
|---|---|
research_topic |
Start deep research on any topic |
mission_status |
Poll progress (phase, quality, claims) |
get_report |
Get the synthesized markdown report |
search_knowledge |
Query claims across all missions |
list_missions |
See completed research history |
git clone https://github.com/qa10devteam/behive.git
cd behive
cp .env.example .env # add your LLM API key
docker compose up -d # API ready at localhost:8091Full stack with knowledge graph + vector search:
docker compose --profile full up -dpip install behive[all]
# PostgreSQL
createdb hive
behive db init
# Configure
export BEHIVE_DB_URL="postgresql://user:pass@localhost:5432/hive"
export BEHIVE_LLM=bedrock # or openai, local
# Start services
behive serve # Starts both REST API (:8091) + MCP (:8090)| Component | Minimum | Recommended |
|---|---|---|
| RAM | 4 GB | 16 GB |
| CPU | 2 cores | 8+ cores |
| Storage | 10 GB | 50 GB |
| GPU | Not required | 4Γ L4 (local LLM) |
| PostgreSQL | 14+ | 16 (pgvector) |
| LLM | Any OpenAI-compatible | Bedrock Claude (Haiku + Sonnet) |
-
You give it a topic. "NVIDIA GPU market 2026"
-
Scout bees plan the research. The Queen decomposes it into 5 axes (market share, financials, products, competition, supply chain). Generates 12-14 search queries per axis. Checks 64 APIs.
-
Harvest bees collect sources. Parallel HTTP fetches ~1000 URLs. HEAD sweep first (fast), then full content extraction on promising ones. Typically lands 60-90 usable documents.
-
Worker bees extract claims. This is where BeHive shines:
- Every document gets parsed into atomic, verifiable claims
- Each claim scored on 5 dimensions (specificity, density, uniqueness, verifiability, structure)
- Claims below 0.55 quality β rejected
- Thin claims (missing dates/numbers) β enriched by Sonnet
- Duplicate claims (Jaccard >0.60) β merged
-
The Queen synthesizes. Claude weaves the verified claims into a structured report with inline citations. No hallucination β every statement maps to a scored claim.
-
Knowledge graph grows. Entities (companies, people, products, amounts) and their relationships are stored in Neo4j. Next research mission on a related topic starts with existing context.
| Variable | Default | Description |
|---|---|---|
BEHIVE_DB_URL |
postgresql://localhost/hive |
PostgreSQL connection |
BEHIVE_LLM |
bedrock |
LLM provider: bedrock, openai, local |
BEHIVE_LLM_URL |
β | Local LLM endpoint (for local mode) |
BEHIVE_NEO4J_URI |
bolt://localhost:7687 |
Neo4j (optional) |
BEHIVE_QDRANT_URL |
http://localhost:6333 |
Qdrant (optional) |
BEHIVE_SCALE |
30 |
Default research scale (30-300) |
BEHIVE_QUALITY_GATE |
0.55 |
Minimum claim quality to store |
AWS_PROFILE |
default |
For Bedrock authentication |
OPENAI_API_KEY |
β | For OpenAI mode |
BeHive tries search backends in priority order and falls through on failure:
| Priority | Backend | Env Variable | Free Tier |
|---|---|---|---|
| 1 | SearXNG (self-hosted) | SEARXNG_URL=http://localhost:8080 |
Unlimited |
| 2 | Brave Search | BRAVE_SEARCH_API_KEY=*** |
2,000 req/month |
| 3 | Serper.dev (Google) | SERPER_API_KEY=*** |
2,500 credits |
| 4 | Tavily | TAVILY_API_KEY=*** |
1,000 req/month |
| 5 | DuckDuckGo | (always available) | Unlimited (slow) |
No env vars set? DDG is the default. Add any key above to instantly upgrade search quality.
| BeHive | GPT-Researcher | Tavily | Perplexity | STORM | |
|---|---|---|---|---|---|
| Output format | Structured JSON | Markdown text | JSON snippets | Text | Wiki article |
| Per-claim scoring | β 0.0-1.0 | β | β | β | β |
| Knowledge graph | β Neo4j | β | β | β | β |
| Cross-session memory | β Cumulative | β | β | β | β |
| MCP native | β | β | β | β | β |
| API sources (64) | β | β Web only | β Web only | ||
| Self-hosted | β Full | β Cloud | β Cloud | β | |
| Quality deduplication | β Jaccard 0.60 | β | β | β | β |
| SSE streaming | β Real-time | β | β | β | β |
| Pricing | Free (MIT) | Free (MIT) | $0.01/search | $20/mo+ | Free (MIT) |
- V4 pipeline (BYOK β bring your own LLM key, any provider)
- Quality scoring (avg 0.77, top missions reach 0.82+)
- REST API (14 endpoints)
- MCP Server (Streamable HTTP)
- SSE streaming (real-time progress)
- Knowledge graph (Neo4j)
- 64 API sources (37 free APIs confirmed working, 26 need BYOK keys)
- Browser search (Chromium/Playwright β scrapes Google/Bing, zero API keys)
-
pip install behive(PyPI) - Docker Compose one-liner
- n8n community node (npm)
- Agent skills (Hermes, OpenClaw, Claude Desktop)
- Web UI dashboard
- Multi-tenant API keys
- Webhook callbacks
- Scheduled recurring research
- PDF export with charts
BeHive works with every major AI agent platform:
| Platform | Method | Install |
|---|---|---|
| Claude Desktop | MCP (zero-code) | Add URL to claude_desktop_config.json |
| Cursor / Windsurf | MCP | Add MCP server in settings |
| Hermes Agent | MCP + Skill | cp integrations/hermes ~/.hermes/skills/research/behive-research |
| OpenClaw | Skill | cp integrations/openclaw ~/.openclaw/workspace/skills/behive-research |
| n8n | Community Node | Install n8n-nodes-behive in Settings β Community Nodes |
| ChatGPT | Custom GPT / API | OpenAPI spec in README above |
| Any MCP client | Streamable HTTP | URL: http://localhost:8090/mcp |
See integrations/ for detailed setup guides.
See CONTRIBUTING.md for development setup, code style, and PR guidelines.
git clone https://github.com/qa10devteam/behive.git
cd behive
pip install -e ".[all,dev]"
pytestMIT β use it, fork it, ship it, sell it.
Built by QA10 Β· Structured knowledge, not text soup.
