Elasticsearch Agent Builder A2A Chatbot

A Python chatbot client for interacting with Elasticsearch Agent Builder using the Agent-to-Agent (A2A) protocol, with built-in benchmarking capabilities.

Features

• Interactive Chat Interface: Real-time conversation with Elasticsearch agents
• A2A Protocol Support: Full implementation of the A2A protocol specification
• Benchmarking Tools: Run multiple chat sessions to measure response times and performance
• Context Management: Maintain conversation context across multiple messages
• Response Time Tracking: Automatic measurement of agent response times
• Statistics & Analytics: Comprehensive performance metrics (min, max, mean, median, standard deviation)
• JSON Export: Save benchmark results for further analysis

Prerequisites

• Python 3.7 or higher
• Elasticsearch instance with Agent Builder enabled (Serverless or Stack 9.2.0+)
• Elasticsearch API key with access to Agent Builder
• Agent ID from your configured agent

Installation

1. Clone or download this repository

2. Install dependencies:

pip install -r requirements.txt

3. Create a .env file based on .env.example:

cp .env.example .env

4. Edit .env and add your Elasticsearch credentials:

ELASTIC_URL=https://your-elasticsearch-instance.elastic.cloud
AGENT_ID=your-agent-id
API_KEY=your-api-key

Usage

Interactive Chat Mode

Start an interactive chat session with your agent:

python chatbot.py

Commands in chat mode:

• Type your message and press Enter to chat
• new - Start a new conversation context
• exit or quit - Exit the chat

Enable debug mode:

python chatbot.py --debug

Debug Mode

If responses aren't working correctly, use the debug script to inspect raw API communication:

python debug_chat.py

This will show:

• The exact request payload being sent
• The complete response structure from Elasticsearch
• Analysis of where the response text should be extracted from

Benchmark Mode

Run performance benchmarks on your agent:

Basic benchmark with custom messages:

python benchmark.py -m "Hello" "How are you?" "Goodbye" -i 10

Using a configuration file:

python benchmark.py -c benchmark_config.json

With shared context across iterations:

python benchmark.py -m "Hello" "Tell me more" --shared-context -i 5

Save results to file:

python benchmark.py -m "Test message" -i 10 -o results.json

Benchmark Configuration File

Create a JSON file to define your benchmark:

{
  "messages": [
    "Hello, how can you help me?",
    "What are your capabilities?",
    "Can you help me with data analysis?",
    "Thank you for the information"
  ],
  "iterations": 10
}

Command-Line Options

benchmark.py options:

• -m, --messages: Messages to send (space-separated)
• -i, --iterations: Number of iterations (default: 5)
• -c, --config: Path to JSON configuration file
• --shared-context: Use same context across all iterations
• -o, --output: Output file for results (JSON format)

Architecture

A2A Client (a2a_client.py)

Core client implementing the A2A protocol:

• JSON-RPC 2.0 message formatting
• HTTP communication with Elasticsearch
• Response time measurement
• Agent card retrieval

Chatbot (chatbot.py)

Interactive chat interface:

• Colored terminal output
• Context management
• Real-time conversation
• Agent information display

Benchmark Runner (benchmark.py)

Performance testing tool:

• Multiple iteration support
• Statistical analysis
• JSON export capability
• Configurable test scenarios

Benchmark Results

Benchmark results include:

Statistics:

• Total requests
• Successful requests
• Min/Max response times
• Mean response time
• Median response time
• Standard deviation
• Total elapsed time

Detailed Results:

• Per-iteration timing
• Individual message responses
• Error tracking

Example Output

Interactive Chat

==========================================
Agent Information:
==========================================
Agent ID: my-agent-123
Name: Customer Support Agent
==========================================

You: Hello, how can you help me?
Agent: I can help you with customer inquiries, product information, and support requests.
(Response time: 1.23s)

Benchmark Results

==========================================
Benchmark Results
==========================================
Total Requests: 30
Successful: 30

Response Times (seconds):
  Min:    0.856s
  Max:    2.134s
  Mean:   1.245s
  Median: 1.198s
  StdDev: 0.312s
  Total:  37.350s
==========================================

A2A Protocol Details

This implementation follows the A2A protocol specification:

• JSON-RPC 2.0: Standard request/response format
• Message Structure: Supports text, file, and data parts
• Context Tracking: Maintains conversation context via contextId
• Error Handling: Proper error code handling per specification

Troubleshooting

Agent only returns description/greeting instead of responding:

This was a common issue caused by incorrect request format. Fixed in latest version - the client now sends messages directly to Elasticsearch without the JSON-RPC 2.0 wrapper (which Elasticsearch doesn't use).

If you still have issues:

1. Update your code - Make sure you're using the latest version where use_jsonrpc=False is the default

2. Run debug script to see raw requests:

   python debug_chat.py

   This will show the exact request/response format being used.

3. Enable debug mode in chatbot:

   python chatbot.py --debug

   Or set in .env:

   DEBUG=true
   

4. Verify the request format:

   • By default, messages are sent directly (not wrapped in JSON-RPC)
   • The request should be just the message object with messageId, role, and parts
   • If you need JSON-RPC format for some reason, set use_jsonrpc=True

Authentication errors:

• Verify your API key is correct
• Ensure the API key has Agent Builder permissions

Connection errors:

• Check your ELASTIC_URL is correct
• Verify network connectivity to Elasticsearch

Agent not found:

• Confirm your AGENT_ID exists
• Check agent is deployed and accessible

Missing dependencies:

pip install -r requirements.txt

Security Notes

• Never commit your .env file to version control
• Keep your API key secure
• Use environment variables for sensitive configuration
• Rotate API keys regularly

License

This project is provided as-is for use with Elasticsearch Agent Builder.

Resources

• Elasticsearch Agent Builder Documentation
• A2A Protocol Specification
• Elasticsearch API Keys