Data Analyst MCP (RAG + Dataset Tools)

This repo contains:

• RAG pipeline: documents/ → FAISS index (faiss_index/) → LLM answers via rag.py
• Dataset utilities: data_analysis.py (CSV/Excel via pandas + charts)
• MCP server: server.py exposes tools for an MCP client
• Web app: webapp/ — a FastAPI backend + simple HTML/JS frontend over the same dataset/RAG functions, for use in a browser instead of an MCP client

---

Prerequisites

• Python 3.10+
• A Groq API key (used by langchain_groq)

---

1) Install dependencies

From the project root:

pip install -r requirements.txt

---

2) Configure environment variables (.env)

Create .env in the project root.

copy .env.example .env

Edit .env and set your Groq API key (variable name must match what langchain_groq expects).

---

3) Add inputs

Documents for RAG

Place .txt or .pdf files in:

• documents/

Datasets for analysis

Place .csv, .xlsx, .xls files in:

• datasets/

---

4) Build the FAISS index (run once, then after doc changes)

python ingest.py

Generates:

• faiss_index/index.faiss
• faiss_index/index.pkl

---

5) Start the MCP server

python server.py

The server exposes tools including:

• answer_question(query) (RAG over documents/)
• list_datasets()
• dataset_info(filename)
• preview_dataset(filename, n)
• summary_statistics(filename)
• filter_dataset(filename, query, limit)
• group_by_aggregate(filename, group_by, agg_column, agg_func)
• correlation_matrix(filename)
• plot_chart(filename, chart_type, x, y=None)

Charts are saved to:

• outputs/

Supported chart_type: bar, line, scatter, hist

---

Web app (FastAPI + simple frontend)

An alternative to using an MCP client: a small FastAPI backend (webapp/main.py) exposes the same dataset tools (and RAG) as HTTP endpoints, with a plain HTML/JS frontend (webapp/static/) to drive them from a browser.

Run

python -m uvicorn webapp.main:app --reload

Then open http://127.0.0.1:8000/ in a browser.

Endpoints

• GET /api/datasets
• GET /api/datasets/{filename}/info
• GET /api/datasets/{filename}/preview?n=5
• GET /api/datasets/{filename}/summary
• GET /api/datasets/{filename}/filter?query=...&limit=20
• GET /api/datasets/{filename}/group_by?group_by=...&agg_column=...&agg_func=mean
• GET /api/datasets/{filename}/correlation
• GET /api/datasets/{filename}/plot?chart_type=bar&x=...&y=... — returns {"url": "/outputs/..."}, an image path served statically
• GET /api/documents/sources, GET /api/ask?query=... — RAG endpoints; rag.py is imported lazily on first use, so the app still starts fine even without a Groq key or FAISS index configured

The frontend has two tabs: Datasets (dropdown + buttons for info/preview/summary/correlation, forms for filter/group-by/chart) and Ask Documents (RAG Q&A).

Note: this web app doesn't go through the MCP protocol — it calls data_analysis.py/rag.py directly. Use it for browser-based / human interaction; use server.py for MCP clients (Claude Desktop, Cursor, etc.).

---

Docker

Build and run the server in a container instead of a local venv.

Build

docker build -t data-analyst-mcp .

This bakes in all dependencies (CPU-only PyTorch, to avoid pulling ~1.3GB of unneeded CUDA packages) and pre-downloads the all-MiniLM-L6-v2 embedding model at build time, since rag.py forces HF_HUB_OFFLINE=1 at runtime.

Run

The server speaks MCP over stdio, so it's meant to be launched by an MCP client (not left running as a background service). Point your client's command at:

docker run --rm -i --env-file .env ^
  -v "%cd%\documents:/app/documents" ^
  -v "%cd%\datasets:/app/datasets" ^
  -v "%cd%\outputs:/app/outputs" ^
  -v "%cd%\faiss_index:/app/faiss_index" ^
  data-analyst-mcp

Or with Docker Compose (docker-compose.yml already wires up the same volumes and .env):

docker compose run --rm data-analyst-mcp

On first run, if faiss_index/index.faiss isn't present, the container runs ingest.py automatically before starting server.py (requires files already in documents/).

Git Bash on Windows users: MSYS path conversion mangles container paths like /app/datasets in -v flags. Prefix the command with MSYS_NO_PATHCONV=1 if invoking docker run from Git Bash instead of cmd/PowerShell.

Rebuilding

Re-run docker build whenever requirements.txt changes. Document/dataset changes don't require a rebuild since those folders are volume-mounted — just re-run ingest.py inside the container (or delete faiss_index/index.faiss so the entrypoint rebuilds it) after editing documents/.

---

Common issues

FAISS not found / mismatch

If you changed documents, re-run:

python ingest.py

Groq authentication errors

Ensure .env exists and contains a valid Groq API key.

---

Quick smoke test

1. Run:

   python ingest.py
   python server.py

2. Call MCP tool list_datasets() from your MCP client.
3. Call MCP tool answer_question("...your question...").