Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Project

The general purpose of mcp-optimizer is to develop a MCP server that acts as an intelligent intermediary between AI clients and multiple MCP servers. The MCP Optimizer server addresses the challenge of managing large numbers of MCP tools by providing semantic tool discovery, caching, and unified access through a single endpoint.
The general purpose of the ToolHive MCP Optimizer is to develop a MCP server that acts as an intelligent intermediary between AI clients and multiple MCP servers. The MCP Optimizer server addresses the challenge of managing large numbers of MCP tools by providing semantic tool discovery, caching, and unified access through a single endpoint.

## Technical considerations
- Use uv as package manager. `uv add <package>` for adding a package. `uv add <package> --dev` for development packages for linting and testing
Expand Down
23 changes: 13 additions & 10 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Contributing to `mcp-optimizer` <!-- omit from toc -->

First off, thank you for taking the time to contribute to mcp-optimizer! :+1: :tada: mcp-optimizer
is released under the Apache 2.0 license. If you would like to contribute
something or want to hack on the code, this document should help you get
started. You can find some hints for starting development in mcp-optimizer's
First off, thank you for taking the time to contribute to the ToolHive MCP
Optimizer! :+1: :tada: MCP Optimizer is released under the Apache 2.0 license.
If you would like to contribute something or want to hack on the code, this
document should help you get started. You can find some hints for starting
development in mcp-optimizer's
[README](https://github.com/StacklokLabs/mcp-optimizer/blob/main/README.md).

## Table of contents <!-- omit from toc -->
Expand All @@ -26,8 +27,8 @@ report unacceptable behavior to

## Reporting security vulnerabilities

If you think you have found a security vulnerability in mcp-optimizer please DO NOT
disclose it publicly until we've had a chance to fix it. Please don't report
If you think you have found a security vulnerability in mcp-optimizer please DO
NOT disclose it publicly until we've had a chance to fix it. Please don't report
security vulnerabilities using GitHub issues; instead, please follow this
[process](https://github.com/StacklokLabs/mcp-optimizer/blob/main/SECURITY.md)

Expand All @@ -52,18 +53,20 @@ are a great place to start!
### Pull request process

- All commits must include a Signed-off-by trailer at the end of each commit
message to indicate that the contributor agrees to the Developer Certificate of
Origin. For additional details, check out the [DCO instructions](dco.md).
message to indicate that the contributor agrees to the Developer Certificate
of Origin. For additional details, check out the [DCO instructions](dco.md).

- Create an issue outlining the fix or feature.
- Fork the mcp-optimizer repository to your own GitHub account and clone it locally.
- Fork the mcp-optimizer repository to your own GitHub account and clone it
locally.
- Hack on your changes.
- Correctly format your commit messages, see
[Commit message guidelines](#commit-message-guidelines) below.
- Open a PR by ensuring the title and its description reflect the content of the
PR.
- Ensure that CI passes, if it fails, fix the failures.
- Every pull request requires a review from the core mcp-optimizer team before merging.
- Every pull request requires a review from the core mcp-optimizer team before
merging.
- Once approved, all of your commits will be squashed into a single commit with
your PR title.

Expand Down
80 changes: 56 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
# MCP-Optimizer
# ToolHive MCP Optimizer

An intelligent intermediary MCP server that provides semantic tool discovery, caching, and unified access to multiple MCP servers through a single endpoint.
An intelligent intermediary MCP server that provides semantic tool discovery,
caching, and unified access to multiple MCP servers through a single endpoint.

## Features

- **Semantic Tool Discovery**: Intelligently discover and route requests to appropriate MCP tools
- **Semantic Tool Discovery**: Intelligently discover and route requests to
appropriate MCP tools
- **Unified Access**: Single endpoint to access multiple MCP servers
- **Tool Management**: Manage large numbers of MCP tools seamlessly
- **Group Filtering**: Filter tool discovery by ToolHive groups for multi-environment support
- **Group Filtering**: Filter tool discovery by ToolHive groups for
multi-environment support

## Requirements for Development

Expand All @@ -16,28 +19,35 @@ An intelligent intermediary MCP server that provides semantic tool discovery, ca

## Usage

MCP-Optimizer server is meant to be run with ToolHive. It will automatically discover the MCP workloads running in ToolHive and their tools.
MCP Optimizer is meant to be run with ToolHive. It will automatically discover
the MCP workloads running in ToolHive and their tools.

### Prerequisites

- [ToolHive UI](https://docs.stacklok.com/toolhive/tutorials/quickstart-ui#step-1-install-the-toolhive-ui) (version >= 0.6.0)
- [ToolHive CLI](https://docs.stacklok.com/toolhive/tutorials/quickstart-cli#step-1-install-toolhive) (version >= 0.3.1)
- [ToolHive UI](https://docs.stacklok.com/toolhive/tutorials/quickstart-ui#step-1-install-the-toolhive-ui)
(version >= 0.6.0)
- [ToolHive CLI](https://docs.stacklok.com/toolhive/tutorials/quickstart-cli#step-1-install-toolhive)
(version >= 0.3.1)

### Setup

ToolHive UI must be running for the setup.

#### 1. Run MCP-Optimizer server in a dedicated group in ToolHive.
#### 1. Run MCP Optimizer in a dedicated group in ToolHive

We need to run MCP Optimizer in a dedicated group to configure AI clients
(Cursor, Claude Desktop, etc.) for this group only. This ensures that clients
are configured with only the MCP Optimizer server, while other MCP servers
remain installed but not configured in the AI clients.

We need to run MCP-Optimizer in a dedicated group to configure AI clients (Cursor, Claude Desktop, etc.) for this group only. This ensures that clients are configured with only the MCP-Optimizer server, while other MCP servers remain installed but not configured in the AI clients.
```bash
# 1. Create the group
thv group create optim
# 2. Run MCP-Optimizer in the dedicated group
# 2. Run MCP Optimizer in the dedicated group
thv run --group optim mcp-optimizer
```

#### 2. Configure MCP-Optimizer server with your favorite AI client.
#### 2. Configure MCP Optimizer with your favorite AI client

```bash
# thv client register <client_name> --group optim. Example:
Expand All @@ -57,8 +67,11 @@ thv ls

### Run

Now you should be able to use MCP-Optimizer in the chat of the configured client. Examples:
Now you should be able to use MCP Optimizer in the chat of the configured
client. Examples:

- With the Github MCP server installed, get a GitHub issue details

```markdown
Get the details of GitHub issue 1911 from stacklok/toolhive repo
```
Expand All @@ -67,14 +80,15 @@ Get the details of GitHub issue 1911 from stacklok/toolhive repo

### Runtime Mode

MCP-Optimizer supports two runtime modes for deploying and managing MCP servers:
MCP Optimizer supports two runtime modes for deploying and managing MCP servers:

- **docker** (default): Run MCP servers as Docker containers
- **k8s**: Run MCP servers as Kubernetes workloads

Configuration is case-insensitive (e.g., `K8S`, `Docker`, `k8s` are all valid).

**Quick Start:**

```bash
# Run in Kubernetes mode via environment variable
export RUNTIME_MODE=k8s
Expand All @@ -87,31 +101,41 @@ mcpo --runtime-mode k8s
mcpo --runtime-mode docker
```

For detailed documentation on runtime modes, including code examples, see [docs/runtime-modes.md](docs/runtime-modes.md).
For detailed documentation on runtime modes, including code examples, see
[docs/runtime-modes.md](docs/runtime-modes.md).

### Group Filtering

MCP-Optimizer supports filtering tool discovery by ToolHive groups. This is useful when running multiple MCP servers across different environments (production, staging, development, etc.) and you want to limit tool discovery to specific groups.
MCP Optimizer supports filtering tool discovery by ToolHive groups. This is
useful when running multiple MCP servers across different environments
(production, staging, development, etc.) and you want to limit tool discovery to
specific groups.

**Quick Start:**

```bash
# Only discover tools from production and staging groups
export ALLOWED_GROUPS="production,staging"
mcp-optimizer
```

For detailed documentation on group filtering, including usage examples, see [docs/group-filtering.md](docs/group-filtering.md).
For detailed documentation on group filtering, including usage examples, see
[docs/group-filtering.md](docs/group-filtering.md).

### Connection Resilience

MCP-Optimizer automatically handles ToolHive connection failures with intelligent retry logic. If ToolHive (`thv serve`) restarts on a different port, MCP-Optimizer will:
MCP Optimizer automatically handles ToolHive connection failures with
intelligent retry logic. If ToolHive (`thv serve`) restarts on a different port,
MCP Optimizer will:

1. **Detect the connection failure** across all ToolHive API operations
2. **Automatically rescan** for ToolHive on the new port
3. **Retry with exponential backoff** (1s → 2s → 4s → ... up to 60s)
4. **Gracefully exit** after exhausting all retries (default: 100 attempts over ~100 minutes)
4. **Gracefully exit** after exhausting all retries (default: 100 attempts over
~100 minutes)

**Configuration:**

```bash
# Customize retry behavior via environment variables
export TOOLHIVE_MAX_RETRIES=150 # Max retry attempts (default: 100)
Expand All @@ -122,13 +146,21 @@ export TOOLHIVE_MAX_BACKOFF=120.0 # Maximum delay in seconds (default:
mcp-optimizer --toolhive-max-retries 15 --toolhive-initial-backoff 2.0
```

This ensures MCP-Optimizer remains operational even when ToolHive restarts, minimizing service interruptions. For detailed information on configuration options, testing scenarios, and troubleshooting, see [docs/connection-resilience.md](docs/connection-resilience.md).
This ensures MCP Optimizer remains operational even when ToolHive restarts,
minimizing service interruptions. For detailed information on configuration
options, testing scenarios, and troubleshooting, see
[docs/connection-resilience.md](docs/connection-resilience.md).

### Environment Variables

- `RUNTIME_MODE`: Runtime mode for MCP servers (`docker` or `k8s`, default: `docker`)
- `ALLOWED_GROUPS`: Comma-separated list of ToolHive group names to filter tool lookups (default: no filtering)
- `TOOLHIVE_MAX_RETRIES`: Maximum retry attempts on connection failure (default: `100`, range: 1-500)
- `TOOLHIVE_INITIAL_BACKOFF`: Initial retry backoff delay in seconds (default: `1.0`, range: 0.1-10.0)
- `TOOLHIVE_MAX_BACKOFF`: Maximum retry backoff delay in seconds (default: `60.0`, range: 1.0-300.0)
- `RUNTIME_MODE`: Runtime mode for MCP servers (`docker` or `k8s`, default:
`docker`)
- `ALLOWED_GROUPS`: Comma-separated list of ToolHive group names to filter tool
lookups (default: no filtering)
- `TOOLHIVE_MAX_RETRIES`: Maximum retry attempts on connection failure (default:
`100`, range: 1-500)
- `TOOLHIVE_INITIAL_BACKOFF`: Initial retry backoff delay in seconds (default:
`1.0`, range: 0.1-10.0)
- `TOOLHIVE_MAX_BACKOFF`: Maximum retry backoff delay in seconds (default:
`60.0`, range: 1.0-300.0)
- Additional configuration options can be found in `src/mcp_optimizer/config.py`
34 changes: 17 additions & 17 deletions docs/connection-resilience.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Connection Resilience

MCP-Optimizer includes robust connection retry logic to handle scenarios where ToolHive (`thv serve`) becomes unavailable or restarts on a different port.
MCP Optimizer includes robust connection retry logic to handle scenarios where ToolHive (`thv serve`) becomes unavailable or restarts on a different port.

## Overview

When ToolHive restarts (e.g., after a crash or manual restart), it may bind to a different port within the configured range. MCP-Optimizer automatically detects connection failures and attempts to rediscover and reconnect to ToolHive, minimizing service interruptions.
When ToolHive restarts (e.g., after a crash or manual restart), it may bind to a different port within the configured range. MCP Optimizer automatically detects connection failures and attempts to rediscover and reconnect to ToolHive, minimizing service interruptions.

## Features

### 1. Automatic Port Rediscovery

When a connection failure is detected, MCP-Optimizer:
When a connection failure is detected, MCP Optimizer:
- Rescans the configured port range (default: 50000-50100)
- Tries the initially configured port first (if specified)
- Updates its internal connection details when a new port is found
Expand All @@ -28,7 +28,7 @@ To avoid overwhelming the system during extended outages:

- Default: 100 retry attempts (approximately 100 minutes)
- Configurable range: 1-500 attempts
- After exhausting retries, MCP-Optimizer logs a critical error and exits gracefully
- After exhausting retries, MCP Optimizer logs a critical error and exits gracefully

### 4. Comprehensive Coverage

Expand Down Expand Up @@ -65,7 +65,7 @@ mcp-optimizer \

### Configuration File

These settings follow the standard MCP-Optimizer configuration hierarchy:
These settings follow the standard MCP Optimizer configuration hierarchy:
1. CLI options (highest priority)
2. Environment variables
3. Default values (lowest priority)
Expand All @@ -78,13 +78,13 @@ These settings follow the standard MCP-Optimizer configuration hierarchy:
# Terminal 1: Start ToolHive
thv serve

# Terminal 2: Start MCP-Optimizer with defaults
# Terminal 2: Start MCP Optimizer with defaults
mcp-optimizer
```

**Behavior:**
- MCP-Optimizer connects to ToolHive
- If ToolHive restarts, MCP-Optimizer retries for ~3-4 minutes
- MCP Optimizer connects to ToolHive
- If ToolHive restarts, MCP Optimizer retries for ~3-4 minutes
- Exits if ToolHive doesn't come back online

### Example 2: Extended Retry Window
Expand Down Expand Up @@ -128,7 +128,7 @@ This tests the automatic port rediscovery feature:
thv serve
# Note the port (e.g., 50001)

# Terminal 2: Start MCP-Optimizer
# Terminal 2: Start MCP Optimizer
mcp-optimizer

# Terminal 1: Kill ToolHive
Expand All @@ -140,7 +140,7 @@ thv serve
```

**Expected Result:**
- MCP-Optimizer detects connection failure
- MCP Optimizer detects connection failure
- Logs: "ToolHive connection failed"
- Logs: "Attempting to rediscover ToolHive port"
- Logs: "Successfully rediscovered ToolHive on new port"
Expand All @@ -154,15 +154,15 @@ This tests the exponential backoff and ultimate failure handling:
# Terminal 1: Start ToolHive
thv serve

# Terminal 2: Start MCP-Optimizer
# Terminal 2: Start MCP Optimizer
mcp-optimizer

# Terminal 1: Kill ToolHive (don't restart)
pkill -f 'thv serve'
```

**Expected Result:**
- MCP-Optimizer detects connection failure
- MCP Optimizer detects connection failure
- Retries with increasing delays
- Logs each attempt with backoff time
- After 100 attempts (~100 minutes), logs critical error
Expand All @@ -176,22 +176,22 @@ This tests initial connection retry:
# Make sure ToolHive is not running
pkill -f 'thv serve'

# Start MCP-Optimizer
# Start MCP Optimizer
mcp-optimizer

# In another terminal, start ToolHive within retry window
thv serve
```

**Expected Result:**
- MCP-Optimizer attempts initial connection
- MCP Optimizer attempts initial connection
- Retries with exponential backoff
- If ToolHive starts during retry window: connects and continues
- If ToolHive doesn't start: exits after max retries

## Logging

MCP-Optimizer provides detailed logging at each stage:
MCP Optimizer provides detailed logging at each stage:

### Connection Failure
```
Expand Down Expand Up @@ -255,15 +255,15 @@ The polling manager also implements connection resilience:

## Troubleshooting

### Problem: MCP-Optimizer exits too quickly
### Problem: MCP Optimizer exits too quickly

**Solution:** Increase retry attempts and/or backoff delays:
```bash
export TOOLHIVE_MAX_RETRIES=20
export TOOLHIVE_MAX_BACKOFF=120.0
```

### Problem: MCP-Optimizer takes too long to fail
### Problem: MCP Optimizer takes too long to fail

**Solution:** Decrease retry attempts:
```bash
Expand Down
Loading