Skip to content

armanjscript/Autogen-MCP-Agent

BranchesTags

Repository files navigation

Autogen-MCP-Agent

A modular, extensible AI assistant that combines AutoGen (ag2) agent orchestration with the Model Context Protocol (MCP) for structured tool exposure, all wrapped in a clean Streamlit chat interface. The agent leverages NVIDIA NIM-hosted LLMs and provides out-of-the-box capabilities for file operations, web search, weather lookups, and location-based place discovery.

Table of Contents

Overview

Autogen-MCP-Agent demonstrates a production-oriented pattern for building LLM-powered agents: instead of hardcoding tool logic inside the agent, tools are served through a separate MCP server process. The AutoGen agent discovers and invokes these tools over the stdio transport using the StdioMcpToolAdapter, keeping concerns cleanly separated and making it straightforward to add, remove, or modify tools independently of the agent itself.

The user interacts with the system through a browser-based Streamlit UI that manages chat sessions, renders rich responses (including inline maps and place cards), and persists conversation history to a local SQLite database.

Features

  • MCP-Based Tool Architecture — Tools are registered on a standalone MCP server and consumed by the AutoGen agent via the StdioMcpToolAdapter, following the official Model Context Protocol specification.
  • Six Built-in Tools — File creation, editing, and deletion; web search via Google Serper; real-time weather via WeatherAPI; and location/place discovery via Google Maps & Places API.
  • Dual AutoGen Backends — Supports the newer autogen-agentchat + autogen-ext stack (preferred, with native MCP integration) and falls back to legacy autogen if the extensions are unavailable.
  • Graceful Fallback — When neither AutoGen backend is present, the agent falls back to direct tool execution with keyword-based intent detection, so the system remains partially functional even without an LLM.
  • Persistent Chat History — All conversations are stored in a local SQLite database with per-session isolation, enabling session switching and history replay.
  • Streamlit Chat UI — A responsive, light-themed chat interface with sidebar session management, special rendering for location results (map images + place cards), and weather emoji display.
  • NVIDIA NIM LLM Backend — Uses the NVIDIA Integrate API endpoint (integrate.api.nvidia.com/v1) as an OpenAI-compatible endpoint, configured via pydantic-settings.
  • Configuration via .env — All API keys and settings are managed through a .env file, loaded automatically by pydantic-settings.

Tech Stack

Layer Technology
LLM Backend NVIDIA NIM (OpenAI-compatible API) — default model: deepseek-ai/deepseek-v4-pro
Agent Framework autogen-agentchat + autogen-ext (primary) / autogen (legacy fallback)
Tool Protocol Model Context Protocol (mcp) via stdio transport
UI Streamlit
Database SQLite (via Python sqlite3)
Configuration pydantic-settings + .env file
HTTP Clients requests (for external APIs)
Geocoding Google Maps Geocoding API + Google Places API
Web Search Google Serper API
Weather WeatherAPI.com
Python Version >= 3.12

Architecture

Architecture Diagram

Autogen-MCP-Agent Architecture

The system is organized into five distinct layers, each with a clear responsibility. Data flows downward from the user's browser through the Streamlit UI into the MasterAgent, which communicates with the MCP server over stdio to invoke tools. The NVIDIA NIM LLM sits alongside the agent as an external service. All conversation data is persisted to a local SQLite database, and configuration is centralized in config/settings.py using pydantic-settings.

Project Structure

Autogen-MCP-Agent/
├── main.py                          # Entry point — launches Streamlit app
├── run.py                           # Alternative entry point with explicit server args
├── pyproject.toml                   # Project metadata and dependencies (uv/pip compatible)
├── env-style.txt                    # Template listing all required environment variables
├── README.md                        # This file
├── architecture.png             # Architecture diagram
│
├── config/
│   ├── __init__.py                  # Exports `settings` singleton
│   └── settings.py                  # Pydantic Settings class — all config in one place
│
└── src/
    ├── agents/
    │   ├── __init__.py
    │   └── master_agent.py          # Core agent: orchestrates LLM, MCP tools, sessions
    │
    ├── mcp/
    │   ├── __init__.py
    │   ├── mcp_server.py            # MCPServerWrapper — FastMCP convenience wrapper (fallback use)
    │   └── http_server.py           # *** THE ACTUAL MCP SERVER *** — mcp.server.Server via stdio
    │
    ├── tools/
    │   ├── __init__.py
    │   ├── file_tools.py            # File CRUD operations (create, edit, delete)
    │   ├── web_search_tools.py      # Google Serper web search
    │   ├── weather_tools.py         # WeatherAPI.com current weather
    │   └── location_tools.py        # Google Geocoding + Places API
    │
    ├── database/
    │   ├── __init__.py
    │   └── db_manager.py            # SQLite manager — sessions, messages, history
    │
    ├── memory/
    │   ├── __init__.py
    │   └── session_manager.py       # In-memory session cache backed by DB
    │
    └── ui/
        ├── __init__.py
        └── streamlit_app.py         # Full Streamlit chat UI with sidebar + rich output

Component Breakdown

config/settings.py

Central configuration using pydantic-settings. Reads from environment variables and .env file. Holds all API keys, the LLM model name, the NVIDIA API base URL, MCP server host/port, and the database URL. A module-level settings singleton is imported throughout the codebase.

src/agents/master_agent.py

The brain of the application. The MasterAgent class:

  1. Creates an OpenAIChatCompletionClient pointed at the NVIDIA NIM endpoint.
  2. Spawns a child Python process running src.mcp.http_server as an MCP server.
  3. Loads each of the six tools individually via StdioMcpToolAdapter.from_server_params().
  4. Creates an AssistantAgent with the loaded tools and a system prompt.
  5. On each user query, prepends conversation history as context, sends the task to the agent, and saves the exchange to the database.

If the new AutoGen extensions are not installed, it falls back to legacy autogen.AssistantAgent + UserProxyAgent. If neither is available, it uses a keyword-matching fallback that calls tools directly.

src/mcp/http_server.py (The Actual MCP Server)

This is the real MCP server used at runtime. It inherits from mcp.server.Server and communicates over stdio transport. This is the process that the AutoGen agent connects to via StdioMcpToolAdapter. It defines the full JSON Schema for each tool's input parameters and dispatches calls to the appropriate tool class through handle_list_tools() and handle_call_tool() handlers. This file must be running (either manually or as an auto-spawned child process) before the agent can use any tools.

src/mcp/mcp_server.py (Convenience Wrapper — Not the Runtime Server)

A higher-level wrapper using FastMCP (the convenience API from the mcp package). It registers the same six tools using the @server.tool() decorator. This wrapper also provides a execute_tool() method for direct (non-MCP) tool invocation, used by the fallback path when AutoGen is not available. This is NOT the server used at runtime — it exists as a utility for direct tool execution and testing.

src/tools/

Four tool classes, each encapsulating a specific capability:

  • FileTools (file_tools.py) — Local filesystem operations with pathlib.Path. Creates, edits, and deletes files with automatic parent directory creation.
  • WebSearchTools (web_search_tools.py) — POST requests to google.serper.dev/search, returning top 5 organic results with titles, links, and snippets.
  • WeatherTools (weather_tools.py) — GET requests to api.weatherapi.com/v1/current.json, returning temperature, conditions, humidity, wind, and "feels like" data with emoji mapping.
  • LocationTools (location_tools.py) — Two-step process: geocode a location string via Google Geocoding API to get lat/lng, then search for nearby places via Google Places API. Also generates a Google Static Maps URL with numbered markers.

src/database/db_manager.py

A lightweight SQLite wrapper using Python's built-in sqlite3. Creates the chat_history table on first run, provides methods for saving messages, retrieving session history, and listing all sessions with message counts.

src/memory/session_manager.py

An optional in-memory cache layer on top of DatabaseManager. Provides create_session(), get_session(), add_message(), and get_session_history() with a configurable limit.

src/ui/streamlit_app.py

The Streamlit frontend. Features include:

  • A sidebar showing all past chat sessions (clickable to switch).
  • A "New Chat" button that creates a fresh session.
  • Custom CSS for a polished, light-themed chat experience.
  • Special rendering for location results: inline Google Static Maps images and place cards with ratings, addresses, and "View Details" buttons.
  • Async bridge: creates a new event loop to call the agent's async process_query() from Streamlit's synchronous context.

How MCP Works in This Project

The Model Context Protocol (MCP) is an open standard that defines how LLM applications expose and consume tools. In this project, MCP serves as the bridge between the tool implementations and the AutoGen agent.

The Real MCP Server: http_server.py

The file src/mcp/http_server.py is the actual MCP server that runs at runtime. Despite its name suggesting HTTP, it uses stdio transport — communication happens over the operating system's standard input and output pipes. This file contains the MCPHttpServer class which inherits from mcp.server.Server and registers two core handlers:

  • handle_list_tools() — Returns a list of types.Tool objects, each with a name, description, and inputSchema (JSON Schema describing the expected parameters). The agent calls this during initialization to discover what tools are available.

  • handle_call_tool(name, arguments) — Dispatches a tool call to the matching tool class method and returns the result as types.TextContent. The agent calls this whenever the LLM decides to invoke a tool.

Important: The other file in the src/mcp/ directory — mcp_server.py — is a convenience wrapper built on FastMCP. It is not the server used at runtime. It exists to provide a execute_tool() method for the fallback path (when AutoGen is not installed) and for testing tools directly without going through the MCP protocol.

Tool Registration & Schema

Every tool is registered with an explicit JSON Schema that tells the LLM exactly what parameters to pass. For example, the create_file tool is registered as:

types.Tool(
    name="create_file",
    description="Create a file with specific extension in a specific path",
    inputSchema={
        "type": "object",
        "properties": {
            "path": {"type": "string", "description": "Directory path"},
            "filename": {"type": "string", "description": "Name of the file"},
            "extension": {"type": "string", "description": "File extension"},
            "content": {"type": "string", "description": "File content", "default": ""}
        },
        "required": ["path", "filename", "extension"]
    }
)

This schema is sent to the LLM during tool discovery, enabling the model to generate properly structured function calls with the correct parameter names and types.

Stdio Transport

The MCP server (http_server.py) communicates with the AutoGen agent over stdio (standard input/output), not HTTP. This is a deliberate choice with several advantages:

  • No network overhead — communication happens over OS pipes, making it fast and secure with no exposed ports.
  • Process isolation — the MCP server runs as a separate child process, so a tool crash doesn't bring down the agent.
  • Simplicity — no need to manage ports, CORS, or network configuration.

The server is started via StdioServerParams in the MasterAgent:

server_params = StdioServerParams(
    command=sys.executable,               # Python interpreter
    args=["-m", "src.mcp.http_server"],   # Run http_server.py as a module
    env=None,                             # Inherit parent environment
)

You can also run the MCP server manually in a separate terminal for debugging:

python -m src.mcp.http_server

Tool Adapter Bridge

AutoGen's autogen-ext package provides StdioMcpToolAdapter, which handles the entire lifecycle:

  1. Launches the MCP server (src/mcp/http_server.py) as a subprocess.
  2. Sends a tools/list request to discover available tools.
  3. Wraps each tool as a callable that AutoGen's AssistantAgent can invoke.
  4. On each invocation, sends a tools/call request to the MCP server and returns the result.

Tools are loaded individually by name to give the agent fine-grained control over which tools are available:

for tool_name in ["create_file", "edit_file", "delete_file",
                   "find_locations", "web_search", "get_weather"]:
    adapter = await StdioMcpToolAdapter.from_server_params(
        server_params,
        tool_name=tool_name
    )
    self.tools.append(adapter)

Tool Calling Flow

Primary Path: AutoGen + MCP

User types message in Streamlit UI
        |
        v
MasterAgent.process_query(query, session_id)
        |
        +-- Load last 10 messages from DB (conversation context)
        +-- Prepend context to the user query
        |
        v
AssistantAgent.run(task=context + query)
        |
        v
LLM (NVIDIA NIM / deepseek-v4-pro) decides to call a tool
        |
        v
StdioMcpToolAdapter sends tools/call to MCP server (via stdio)
        |
        v
MCP Server (http_server.py) dispatches to the appropriate Tool class
        |
        v
Tool class executes (e.g., WeatherTools.get_weather("Tokyo"))
        |
        v
Result flows back: Tool --> MCP Server --> Adapter --> Agent --> LLM
        |
        v
LLM generates a natural language response using the tool result
        |
        v
Response saved to SQLite and displayed in Streamlit

Fallback Path: Direct Execution

When AutoGen is not available, the agent uses keyword-based intent detection to match user queries to tools:

if "create file" in query_lower:
    params = self._extract_file_params(query)
    return mcp_wrapper.execute_tool("create_file", **params)
elif "weather" in query_lower:
    params = self._extract_weather_params(query)
    return mcp_wrapper.execute_tool("get_weather", **params)
# ... etc

The MCPServerWrapper.execute_tool() method (from mcp_server.py) bypasses MCP entirely and calls the tool classes directly. This ensures the system remains partially functional even without an LLM backend, though tool parameter extraction is less sophisticated (regex-based vs. LLM-driven).

Installation

Prerequisites

  • Python 3.12 or later — the project requires Python >= 3.12 as specified in pyproject.toml.
  • NVIDIA API Key — obtain one from NVIDIA NIM (free tier available).
  • API keys for external tools (optional, based on which tools you want to use):

Step 1 — Clone the Repository

git clone https://github.com/armanjscript/Autogen-MCP-Agent.git
cd Autogen-MCP-Agent

Step 2 — Install uv

uv is a fast Python package and project manager written in Rust. It replaces pip, virtualenv, and pip-tools with a single, significantly faster tool.

macOS / Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows (PowerShell):

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Via pip (any platform):

pip install uv

Via Homebrew (macOS):

brew install uv

Verify the installation:

uv --version

Step 3 — Create a Virtual Environment with uv

From the project root directory, run:

uv venv

This creates a .venv/ directory containing an isolated Python environment. If you want to specify a Python version explicitly:

uv venv --python 3.12

Step 4 — Activate the Virtual Environment

macOS / Linux:

source .venv/bin/activate

Windows (Command Prompt):

.venv\Scripts\activate.bat

Windows (PowerShell):

.venv\Scripts\Activate.ps1

You should see (.venv) prefixed to your terminal prompt, indicating the virtual environment is active.

Step 5 — Install Dependencies

With the virtual environment activated, install all project dependencies from pyproject.toml:

uv pip install -e .

Or, alternatively, using uv sync (which also creates a lockfile):

uv sync

This installs all required packages including:

  • ag2[openai] and autogen-agentchat — AutoGen agent framework
  • autogen-ext[mcp] — AutoGen MCP extensions (includes StdioMcpToolAdapter)
  • mcp[cli] — Model Context Protocol SDK
  • streamlit — Web UI framework
  • pydantic-settings and python-dotenv — Configuration management
  • requests — HTTP client for external APIs
  • geopy — Geocoding utilities
  • sqlalchemy — Database toolkit (available but not actively used; the project uses raw sqlite3)

Step 6 — Configure Environment Variables

Copy the environment template and fill in your API keys:

cp env-style.txt .env

Edit the .env file with your actual keys:

NVIDIA_API_KEY=nvapi-xxxxxxxxxxxxxxxxxxxxxxxx
GOOGLE_SERPER_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxx
WEATHER_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxx
DATABASE_URL=sqlite:///./chat_history.db
GOOGLE_MAPS_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxx
Variable Required Description
NVIDIA_API_KEY Yes API key for NVIDIA NIM (LLM backend)
GOOGLE_SERPER_API_KEY No API key for Google Serper web search
WEATHER_API_KEY No API key for WeatherAPI.com
GOOGLE_MAPS_API_KEY No API key for Google Maps & Places API
DATABASE_URL No SQLite connection string (defaults to sqlite:///./chat_history.db)

Note: The NVIDIA_API_KEY is the only strictly required key for the LLM to function. The other keys are needed only if you want to use the corresponding tools. The system will return an informative error message if a tool is called without its required API key.

Step 7 — Start the MCP Server (Required First!)

Before running the main application, the MCP server (src/mcp/http_server.py) must be running. This is the process that exposes all six tools (file operations, web search, weather, location) to the agent via the Model Context Protocol over stdio.

Open a separate terminal, navigate to the project root, ensure your virtual environment is activated, and run:

python -m src.mcp.http_server

This starts the MCP server in stdio mode, listening for tools/list and tools/call requests. Leave this terminal open — the server will stay running and wait for input.

How it works at runtime: When the MasterAgent initializes, its StdioMcpToolAdapter automatically spawns python -m src.mcp.http_server as a child process behind the scenes. However, running it manually first is recommended so you can see the server's output and debug any tool registration issues. If you skip this step, the adapter will attempt to spawn it automatically — but if there are import errors or configuration issues, they will be silently swallowed.

Step 8 — Run the Application

With the MCP server running (Step 7), open a new terminal (or use the same one after stopping the manual server), activate the virtual environment, and launch the Streamlit app:

Option A — Using the main entry point:

python main.py

Option B — Using the run script (explicit server args):

python run.py

Option C — Running Streamlit directly:

streamlit run src/ui/streamlit_app.py --server.port=8501 --server.address=localhost

The application will be available at http://localhost:8501 in your browser.

Summary of the two-terminal workflow:

Terminal 1 (MCP Server) Terminal 2 (Application)
python -m src.mcp.http_server python run.py
(keeps running, waits for stdio) (launches Streamlit at :8501)

Configuration Reference

All configuration is managed in config/settings.py via the Settings class (powered by pydantic-settings). Values are resolved in this order: environment variables > .env file > defaults.

Setting Default Description
NVIDIA_API_KEY "" NVIDIA NIM API key
NVIDIA_API_BASE https://integrate.api.nvidia.com/v1 NVIDIA API base URL (OpenAI-compatible)
LLM_MODEL deepseek-ai/deepseek-v4-pro LLM model identifier
DATABASE_URL sqlite:///./chat_history.db Database connection string
GOOGLE_SERPER_API_KEY "" Google Serper API key
GOOGLE_MAPS_API_KEY "" Google Maps API key
WEATHER_API_KEY "" WeatherAPI.com key
MCP_SERVER_HOST localhost MCP server host (reserved for future use)
MCP_SERVER_PORT 8022 MCP server port (reserved for future use)

To change the LLM model, either set the LLM_MODEL environment variable or edit config/settings.py. Any OpenAI-compatible model served by NVIDIA NIM can be used.

Available Tools

File Tools

Tool Parameters Description
create_file path, filename, extension, content Creates a new file at the specified path. Parent directories are created automatically if they don't exist.
edit_file path, filename, content, extension Writes content to an existing file, or creates it if extension is provided and the file doesn't exist. If no extension is given, it searches for a file matching filename.* in the given path.
delete_file path, filename Deletes a file at path/filename. Returns an error if the file doesn't exist.

Web Search Tool

Tool Parameters Description
web_search query Searches the web using the Google Serper API. Returns the top 5 organic results with title, link, snippet, and position.

Weather Tool

Tool Parameters Description
get_weather city Fetches the current weather for a city from WeatherAPI.com. Returns temperature (C and F), condition text with emoji, humidity, wind speed, and "feels like" temperature.

Location Tools

Tool Parameters Description
find_locations query, location Two-step process: (1) geocodes the location string via Google Geocoding API to get lat/lng, (2) searches for nearby places matching query via Google Places API. Returns place names, addresses, ratings, coordinates, types, and a Google Static Maps URL with numbered markers.

Session & Memory Management

The application maintains a persistent record of all conversations using SQLite. Key behaviors:

  • Session Isolation — Each chat session is identified by a unique UUID. Messages from different sessions are stored separately and never mixed.
  • Context Window — The agent loads the last 10 messages from the current session to build context for each new query. This gives the LLM conversational awareness without overwhelming the context window.
  • Session Switching — The Streamlit sidebar lists all past sessions with their creation time and message count. Clicking a session loads its full history into the UI.
  • New Chat — The "New Chat" button creates a fresh session UUID and clears the in-memory message list. The previous session's history remains in the database.
  • In-Memory Cache — The SessionManager class optionally caches session data in a Python dictionary for faster access, falling back to the database for uncached sessions.

Database Schema

The project uses a single SQLite table:

CREATE TABLE IF NOT EXISTS chat_history (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    session_id TEXT NOT NULL,
    human_message TEXT,
    ai_message TEXT,
    timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX IF NOT EXISTS idx_session_id ON chat_history(session_id);
  • session_id — UUID identifying the conversation session.
  • human_message — The user's input (NULL if the row represents a system/agent-initiated message).
  • ai_message — The agent's response (NULL if the row represents a user-only turn).
  • timestamp — Auto-set to the insertion time.

The database file (chat_history.db) is created automatically in the project root on first run.

Development

To extend the system with a new tool:

  1. Create the tool class in src/tools/my_tool.py:
from typing import Dict, Any

class MyTool:
    def do_something(self, param: str) -> Dict[str, Any]:
        # Your logic here
        return {"success": True, "result": "..."}

  1. Register it in the MCP server — Add a new entry in src/mcp/http_server.py (low-level types.Tool registration in handle_list_tools()) and add the dispatch case in handle_call_tool(). Also add it to src/mcp/mcp_server.py (FastMCP @server.tool() decorator).

  2. Add it to the tool loading list in src/agents/master_agent.py:

tool_names = [
    # ... existing tools ...
    "do_something"
]

  1. Update the tool map in MCPServerWrapper.execute_tool() in src/mcp/mcp_server.py.

  2. Export from src/tools/__init__.py.

License

This project is provided as-is. Please refer to the original repository for license information.

About

The autogen mcp agent that can handle file managements, weather status of cities and searching the places

Topics

agent mcp sqlite-database chatbot openai sqlite3 file-management autogen chat-history-database streamlit google-console langchain nvidia-nim agentic-ai tool-calling mcp-server google-serper

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages