LLMAgent: Lightweight MCP Integration

[madawei2699]

LLMAgent: Lightweight MCP Integration

Author: LLMAgent Team
Date: March 30, 2025
Version: 0.1.0
Status: Design Proposal

1. Vision and Purpose

LLMAgent's Machine-to-Computer Protocol (MCP) integration aims to provide a simple, lightweight approach for enabling dynamic workflow creation in LLM-powered agents. This integration will allow LLMs to not only make decisions within predefined workflows but also generate new handlers and tools at runtime, enhancing the agent's adaptability and capabilities.

Rather than implementing the full MCP specification with all its complexity, LLMAgent will focus on a minimal subset that enables core dynamic capabilities while remaining easy to understand and implement, making it suitable for personal development projects.

1.1 Core Capabilities

• Code Generation: Allow LLMs to generate handlers and tools as simple code strings
• Safe Execution: Provide basic isolation for running generated code
• Dynamic Loading: Implement a lightweight system for loading new handlers at runtime
• Tool Discovery: Create a simple mechanism for agents to discover available tools
• Code Templates: Offer pre-built templates to guide LLMs in code generation

2. Architecture Overview

┌───────────────────────────────────────────────────────┐
│                  LLMAgent Core                         │
│  ┌─────────────┐    ┌───────────┐    ┌───────────┐    │
│  │ Signals     │    │ Handlers  │    │ Store     │    │
│  └─────────────┘    └───────────┘    └───────────┘    │
└───────────────────────────────────────────────────────┘
                          │
                          ▼
┌───────────────────────────────────────────────────────┐
│             LLMAgent MCP Extension                     │
│                                                        │
│  ┌────────────┐   ┌───────────────┐  ┌─────────────┐  │
│  │ Code       │   │ Simple        │  │ Tool        │  │
│  │ Executor   │   │ Loader        │  │ Registry    │  │
│  └────────────┘   └───────────────┘  └─────────────┘  │
│                                                        │
└───────────────────────────────────────────────────────┘

3. Core Components

3.1 LLMAgent.MCP Module

The main entry point for MCP functionality, which:

• Coordinates between other MCP components
• Provides simple APIs for executing code and loading handlers
• Offers code templates for common use cases
• Integrates with the existing LLMAgent signal-based architecture

3.2 Code Executor

Responsible for safely executing dynamically generated code:

defmodule LLMAgent.MCP.CodeExecutor do
  @doc """
  Executes code string in a simple sandbox environment.
  Returns {:ok, result} or {:error, reason}
  """
  def execute(code_string, context \\ %{}) do
    try do
      {result, _bindings} = Code.eval_string(code_string, [context: context], __ENV__)
      {:ok, result}
    rescue
      e -> {:error, %{reason: :execution_error, message: Exception.message(e)}}
    end
  end
end

3.3 Simple Loader

Handles loading dynamically generated handlers:

defmodule LLMAgent.MCP.SimpleLoader do
  @doc """
  Loads a handler function from a code string.
  Returns {:ok, handler_fn} or {:error, reason}
  """
  def load_handler(code_string) do
    module_name = "DynamicHandler#{:erlang.unique_integer([:positive])}"
    full_code = """
    defmodule #{module_name} do
      #{code_string}
    end
    """
    
    try do
      {module, _} = Code.eval_string(full_code)
      handler_fn = &module.handle/2
      {:ok, handler_fn}
    rescue
      e -> {:error, %{reason: :load_error, message: Exception.message(e)}}
    end
  end
end

3.4 Tool Registry

Manages a registry of dynamically created tools:

defmodule LLMAgent.MCP.ToolRegistry do
  @doc """
  Initializes a simple tool registry
  """
  def new do
    %{}
  end
  
  @doc """
  Registers a tool with a simple description
  """
  def register(registry, name, description, handler_fn) do
    Map.put(registry, name, %{
      description: description,
      handler: handler_fn
    })
  end
  
  @doc """
  Lists all available tools with descriptions
  """
  def list(registry) do
    Enum.map(registry, fn {name, details} ->
      %{name: name, description: details.description}
    end)
  end
  
  @doc """
  Gets a specific tool by name
  """
  def get(registry, name) do
    Map.get(registry, name)
  end
end

4. Integration with LLMAgent

4.1 MCP Signal Types

To support the MCP functionality, we'll add new signal types to LLMAgent.Signals:

• :code_generation - Request for the LLM to generate code
• :code_execution - Signal to execute generated code
• :tool_creation - Signal to create a new tool
• :handler_creation - Signal to create a new handler

4.2 MCP Handlers

New handlers will be added to process the MCP-specific signals:

• code_generation_handler/2 - Prompts the LLM to generate code
• code_execution_handler/2 - Executes generated code using CodeExecutor
• tool_creation_handler/2 - Creates and registers new tools
• handler_creation_handler/2 - Creates and loads new handlers

4.3 Flow Integration

The MCP functionality will be available through a specialized flow:

{flow, state} = LLMAgent.Flows.mcp_enabled_conversation(
  "You are an assistant that can create and use custom tools.",
  existing_tools,
  %{allow_code_generation: true}
)

5. Usage Examples

5.1 LLM-Generated Handler Example

# LLM generates a handler function as a string
handler_code = """
def handle(signal, state) do
  case signal.type do
    :user_query ->
      # Extract entities from user query
      entities = extract_entities(signal.data)
      
      # Store entities in state
      new_state = AgentForge.Store.put(state, :entities, entities)
      
      # Create signal with extracted entities
      entity_signal = AgentForge.Signal.new(:entities, entities)
      {:emit, entity_signal, new_state}
      
    _ ->
      {:skip, state}
  end
end

# Simple entity extraction function
defp extract_entities(text) do
  # Look for dates
  dates = Regex.scan(~r/\\d{1,2}[\\/\\-]\\d{1,2}[\\/\\-]\\d{2,4}/, text)
  
  # Look for locations (simple example)
  locations = Regex.scan(~r/in ([A-Z][a-z]+)/, text)
  |> Enum.map(fn [_, location] -> location end)
  
  # Look for money amounts
  amounts = Regex.scan(~r/\\$\\d+(\\.\\d{2})?/, text)
  |> Enum.map(fn [amount | _] -> amount end)
  
  %{
    dates: dates,
    locations: locations,
    amounts: amounts
  }
end
"""

# Load the handler
{:ok, entity_handler} = LLMAgent.MCP.SimpleLoader.load_handler(handler_code)

# Add to a flow
flow = AgentForge.Flow.new()
|> AgentForge.Flow.add(entity_handler)

5.2 LLM-Generated Custom Tool

# LLM generates tool code
tool_code = """
def sentiment_analysis(text) do
  # Very simple sentiment analysis
  positive_words = ["good", "great", "excellent", "happy", "positive"]
  negative_words = ["bad", "terrible", "sad", "negative", "poor"]
  
  words = String.downcase(text) |> String.split(~r/\\W+/)
  
  positive_count = Enum.count(words, &(Enum.member?(positive_words, &1)))
  negative_count = Enum.count(words, &(Enum.member?(negative_words, &1)))
  
  cond do
    positive_count > negative_count -> :positive
    negative_count > positive_count -> :negative
    true -> :neutral
  end
end
"""

# Execute and register the tool
{:ok, _} = LLMAgent.MCP.CodeExecutor.execute(tool_code)

# Create a handler that uses this dynamically defined function
sentiment_handler = fn signal, state ->
  if signal.type == :text_input do
    sentiment = sentiment_analysis(signal.data)
    new_state = AgentForge.Store.put(state, :sentiment, sentiment)
    sentiment_signal = AgentForge.Signal.new(:sentiment, sentiment)
    {:emit, sentiment_signal, new_state}
  else
    {:skip, state}
  end
end

# Register in tool registry
registry = LLMAgent.MCP.ToolRegistry.new()
|> LLMAgent.MCP.ToolRegistry.register(
  "sentiment_analyzer",
  "Analyzes text sentiment as positive, negative, or neutral",
  sentiment_handler
)

5.3 Dynamic Flow Creation

# LLM receives a task description
task = "Build a workflow that collects tweets about a product, analyzes sentiment, and summarizes results"

# LLM generates flow definition
flow_code = """
# Define the flow based on available tools
def create_product_sentiment_flow(product_name) do
  import AgentForge.Primitives
  
  [
    tool("twitter_searcher", %{query: product_name, limit: 100}),
    transform("$.tweets", "extract_text"),
    loop("$.tweet_texts", [
      tool("sentiment_analyzer", %{text: "$.item"})
    ]),
    tool("results_aggregator", %{sentiments: "$.sentiments"}),
    notify("$.summary", ["email", "dashboard"])
  ]
end
"""

# Execute the flow creation function
{:ok, _} = LLMAgent.MCP.CodeExecutor.execute(flow_code)

# Use the dynamically created flow
flow_def = create_product_sentiment_flow("AgentForge")

# Create flow with primitive handler
flow = AgentForge.Flow.new()
|> AgentForge.Flow.add(&AgentForge.PrimitiveHandler.handle/2)

# Start the flow
AgentForge.Runtime.start(flow, flow_def)

6. Security Considerations

The MCP implementation must address several security concerns:

1. Code Execution Risks: Dynamic code execution can pose security risks. We'll implement:

   • Timeout limits for code execution
   • Memory/CPU usage limits
   • Restricted module access

2. Validation: All LLM-generated code should be validated before execution:

   • Syntax checking
   • Unsafe operation detection
   • Resource usage estimation

3. Isolation: Although complete isolation isn't possible in Elixir:

   • Run code in separate processes
   • Implement simple permission systems
   • Consider optional external sandbox for critical applications

4. Recommended Practices:

   • Review LLM-generated code before execution in production
   • Limit MCP capabilities based on trust level of LLM
   • Use templates to guide code generation

7. Implementation Plan

Phase 1: Core Components

1. Implement CodeExecutor with basic safety mechanisms
2. Implement SimpleLoader for handler loading
3. Implement ToolRegistry for tool management
4. Add new signal types and handlers to LLMAgent

Phase 2: Integration & Templates

1. Create code templates for common patterns
2. Implement MCP-enabled conversation flow
3. Add configuration options for MCP capabilities
4. Document API and usage patterns

Phase 3: Advanced Features

1. Add more advanced code validation
2. Implement resource usage monitoring
3. Develop pattern library for dynamic workflows
4. Create example applications

8. Conclusion

This lightweight MCP integration for LLMAgent provides a powerful yet simple mechanism for enabling dynamic code generation and execution in LLM-powered agents. By focusing on a minimal, easy-to-understand implementation, it makes these capabilities accessible for personal development projects while maintaining the clean architecture and flexibility of the LLMAgent framework.

The design prioritizes:

• Simplicity: Clear, minimal APIs that are easy to understand
• Safety: Basic protections against common risks
• Integration: Seamless compatibility with existing LLMAgent components
• Flexibility: Support for various dynamic code generation scenarios

This approach allows developers to experiment with dynamic agent capabilities without requiring complex infrastructure or expertise in agent systems, aligning with LLMAgent's philosophy of making advanced agent concepts accessible and practical.

[madawei2699]

References

• https://github.com/punkpeye/awesome-mcp-servers
• https://openai.github.io/openai-agents-python/mcp/