Deep Research Agent

A document-driven agentic AI research system that helps conduct comprehensive analysis through persistent context management and tool integration.

Core Philosophy

Document-Centric Memory: Uses persistent documents to maintain context and track progress, solving the fundamental context window limitation of language models.
Structured Communication: Documents all information in a shared scratchpad, ensuring no critical information is lost.
Tool Augmentation: Leverages specialized tools for up-to-date information gathering and analysis.
User Agency: Acts as a collaborative partner, keeping users in control through clear documentation and decision points.

Getting Started

# Install dependencies
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Install Playwright browsers
playwright install chromium

# Set API keys
export OPENAI_API_KEY=your_openai_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key

# Run a research query
python3 deep_research_agent.py "your research query"

Available Tools

Web Search & Analysis:
- Semantic search using DuckDuckGo
- Dynamic web content scraping with Playwright for JavaScript-heavy sites
- Content analysis with source attribution
File Operations: Create/edit files and execute scripts with user confirmation
Package Management: Install and manage Python packages with user confirmation
Terminal Commands: Execute system commands with user confirmation

Example Usage

Analyzing NVIDIA's recent stock performance:

python3 deep_research_agent.py "Perform a detailed analysis on the recent trend of NVDA stock. How did the stock price change? What might have caused it? How about the market sentiment?"

The agent:

Creates a research plan in scratchpad.md

Generates and executes analysis script (with user confirmation):

Confirm execution of command: python3 nvda_analysis.py
Explanation: Executing Python script: nvda_analysis.py
[y/N]:

Produces comprehensive analysis in nvda_analysis_report.md

Example outputs in examples/:

Features

Dynamic Web Scraping: Uses Playwright for reliable scraping of modern web applications with JavaScript rendering
Concurrent Processing: Efficiently handles multiple URLs in parallel
Smart Content Extraction: Filters out noise and extracts meaningful content with proper formatting
Error Handling: Robust error handling with detailed logging for debugging

Contributing

Contributions welcome! Please feel free to submit issues and pull requests.

License

MIT License - see LICENSE file for details.

Built on practical experience with context window management and real-world research needs.

License

grapeot/deep_research_agent

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Languages

Error
Looks like something went wrong!

Packages