🔎 Deep Research — a multi-agent research assistant

Give it a topic. It interviews you with a few clarifying questions, plans and runs web searches, writes a long cited report, grades its own work and re-researches the weak spots, then emails you the result — all from a small Gradio web app.

This repo is meant to be read and tinkered with. It's a compact, end-to-end example of an agentic workflow: several small, single-purpose AI agents coordinated by an orchestrator, with tool use, structured hand-offs, parallel work, a self-correction loop, and human-in-the-loop checkpoints. Every piece is a short, readable Python file.

---

What it does

topic ─▶ clarifying questions ─▶ your answers ─▶ search plan (you approve)
      ─▶ web searches ─▶ written report ─▶ quality evaluation ⟲ (deepen if weak)
      ─▶ email + on-screen report (with sources)

The result is a multi-page markdown report with a ## References section of real source URLs, emailed to you and shown in the app (with a downloadable .md).

The agents (6 sub-agents + 1 orchestrator)

Each agent is a small Agent(...) in its own file; the orchestrator (research_manager.py) is plain Python that calls them in order and runs the loops.

#	Agent	File	Job	Output
—	ResearchManager (orchestrator)	research_manager.py	Coordinates everything; runs the search and deepening loops	streamed status + final report
1	Clarifier	clarifier_agent.py	Turns a topic into N clarifying questions	ClarifyingQuestions
2	Planner	planner_agent.py	Proposes a set of web searches (adaptive count)	WebSearchPlan
3	Search (×N, parallel)	search_agent.py	Searches the web for one term, summarizes it, keeps source URLs	SearchResult{summary, sources}
4	Writer	writer_agent.py	Synthesizes all findings into a cited report	ReportData{summary, report, follow-ups}
5	Evaluator	evaluator_agent.py	Grades the report; gives actionable feedback	EvaluationResult{passes, score, feedback}
6	Email	email_agent.py	Converts the report to HTML and sends it	sends via SendGrid

---

Prerequisites

• Python 3.10+ (the code uses X | None type annotations).
• An OpenAI API key — required. Powers every agent and the web search. Running costs money (multiple LLM calls + live web searches per report).
• A SendGrid account — optional. Only needed to email the report. Without it the app still runs, shows and lets you download the report, and just skips the email step with a notice.

Setup

1. Create and activate a virtual environment:

   python -m venv .venv
   source .venv/bin/activate        # Windows: .venv\Scripts\activate

2. Install dependencies:

   pip install -r requirements.txt

3. Copy the example env file and fill in your own values:

   cp .env.example .env

   Variable	Required	Purpose
   OPENAI_API_KEY	✅	LLM calls + web search (the agents SDK)
   SENDGRID_API_KEY	optional	Sending the report by email
   FROM_EMAIL	for email	A SendGrid-verified sender address
   TO_EMAIL	for email	Default recipient (editable in the UI)

Run

python deep_research.py

The Gradio app opens in your browser. Enter a topic → Get clarifying questions → answer them → Plan searches → review/edit the searches → Approve & Run Research. Progress streams on the left; the finished report renders on the right and is emailed to you.

---

Configuration options

Most knobs are exposed without editing code:

In the UI (per run):

• Number of clarifying questions — slider (1–5).
• Max web searches — slider (1–10); the planner picks how many it actually needs, up to this.
• Email the report to — recipient address (defaults to TO_EMAIL).
• Editable questions and search terms — you can reword the clarifier's questions and the planner's searches, or clear a search box to drop it.

Via environment variables — override the model per agent (defaults to gpt-4o-mini): PLANNER_MODEL, SEARCH_MODEL, CLARIFIER_MODEL, WRITER_MODEL, EVALUATOR_MODEL, EMAIL_MODEL. For example, put WRITER_MODEL=gpt-4o in .env to upgrade only the writer. (See models.py.)

In code — constants worth knowing:

Constant	File	Meaning	Default
HOW_MANY_QUESTIONS	clarifier_agent.py	Default clarifying-question count	3
HOW_MANY_SEARCHES	planner_agent.py	Default max searches	5
MAX_DEEPEN_ROUNDS	research_manager.py	Re-research rounds if the report fails	2
DEEPEN_SEARCHES	research_manager.py	New searches per deepening round	3
SEARCH_ATTEMPTS	research_manager.py	Retries per failing search	2
MAX_QUESTION_BOXES / MAX_SEARCH_BOXES	deep_research.py	UI capacity (slider caps)	5 / 10

---

How it works (the 3-step UI)

1. Topic & questions. You enter a topic; the Clarifier asks a few questions so the research is targeted. You can edit the questions and add answers.
2. Plan. The Planner turns the topic + your answers into a list of web searches. You review and edit them — nothing hits the web until you approve.
3. Run. The orchestrator runs the searches in parallel, the Writer drafts a cited report, the Evaluator grades it, and if it falls short the system deepens (plans extra searches from the feedback, re-researches, and rewrites — up to MAX_DEEPEN_ROUNDS). Then it emails and shows the report.

👉 For a stage-by-stage internal trace (data shapes, the streaming trick, caching, persistence), see FLOW.md.

The agentic workflow — patterns at play

This project is small on purpose so the patterns stand out. If you're learning agent design, here's what to notice:

• Orchestrator + workers. ResearchManager is the orchestrator. It doesn't "think" — it's ordinary control flow that delegates each step to a specialized agent. Small, single-responsibility agents are easier to prompt, test, and swap than one mega-agent.
• Sequential decomposition (prompt chaining). A hard task (write a researched report) is broken into a chain — clarify → plan → search → write → evaluate — where each step's output is the next step's input. Each agent only has to be good at one thing.
• Structured outputs as typed hand-offs. Agents don't pass free text around. Each returns a Pydantic model (WebSearchPlan, SearchResult, ReportData, EvaluationResult) via the SDK's output_type, so the orchestrator gets validated, typed data. Inputs are built from typed dataclasses in payloads.py. This is what makes the hand-offs reliable.
• Tool use. The Search agent calls a hosted WebSearchTool; the Email agent calls a custom @function_tool. The agent loop (Runner.run) lets an agent call tools repeatedly until it produces its final structured answer.
• Parallelization. Independent searches run concurrently with asyncio, so N searches take roughly the time of the slowest one, not the sum.
• Evaluator–optimizer loop (reflection / self-correction). The Evaluator is an LLM-as-judge. When it fails a draft, its feedback is fed back to re-plan research and rewrite — the system improves its own output instead of shipping the first draft. This is the heart of the "deep" in deep research.
• Human-in-the-loop. Two checkpoints (answer the questions, approve the search plan) keep a person in control of scope and spend before the expensive web work runs.
• Efficiency guards. A cache + dedup avoid repeat web calls; retries with backoff absorb transient failures; every run is persisted to runs/ for later inspection.

---

Play around with it (ideas for learning)

• Swap a model. Set EVALUATOR_MODEL=gpt-4o and watch the grading get stricter; or downgrade the writer and see quality drop and the deepening loop kick in.
• Tune the loop. Change MAX_DEEPEN_ROUNDS or DEEPEN_SEARCHES in research_manager.py and observe more/less re-research on hard topics.
• Edit an agent's brain. Each agent's behavior is just its INSTRUCTIONS string — try making the Clarifier ask sharper questions, or the Writer adopt a different tone.
• Make the evaluator picky. Tighten the Evaluator instructions (e.g. "fail anything without ≥5 distinct sources") and see the loop respond.
• Inspect what happened. Open the JSON in runs/, and follow the OpenAI trace URL printed at the start of each run to see every agent call and tool invocation.
• Add a stage. Try inserting a fact-checking agent between Writer and Evaluator — a great exercise in extending an orchestrated pipeline.

Project structure

deep_research/
├── deep_research.py        # Gradio UI + 3-step flow (run this)
├── research_manager.py     # Orchestrator: search/deepen/evaluate/email loops
├── clarifier_agent.py      # Agent 1: clarifying questions
├── planner_agent.py        # Agent 2: search plan
├── search_agent.py         # Agent 3: web search + summary + sources
├── writer_agent.py         # Agent 4: cited report
├── evaluator_agent.py      # Agent 5: quality grader
├── email_agent.py          # Agent 6: SendGrid email
├── payloads.py             # Typed inputs for each agent hand-off
├── models.py               # Per-agent model config (env-overridable)
├── cache.py                # On-disk search cache + dedup
├── storage.py              # Saves each run to runs/
├── conftest.py, tests/     # Offline unit tests (mocked agents)
├── evals/                  # Quality eval harness over fixed topics
├── FLOW.md                 # Deep dive on the internal flow
├── .env.example            # Copy to .env and fill in
├── requirements.txt        # Runtime dependencies
└── requirements-dev.txt    # + pytest for tests

Tests & evals

Run the unit tests (mocked agents — no network/API needed):

pip install -r requirements-dev.txt
pytest

Run the quality eval harness over a fixed set of topics (makes real LLM/web-search calls; needs OPENAI_API_KEY, skips email). It records evaluator scores to evals/results.csv:

python evals/run_evals.py --max-searches 3

Notes & caveats

• Cost: each run makes multiple LLM calls plus live web searches, so it consumes OpenAI credits. The sliders and --max-searches help you keep runs cheap while experimenting.
• Email is real: a successful run sends an actual email via SendGrid to the recipient. No SendGrid config → the app skips email gracefully and still shows/downloads the report.
• Built on the OpenAI Agents SDK (openai-agents) with a Gradio front end.