MCP Enhanced TestExtender

An automated test readability improvement system that uses Model Context Protocol (MCP) and Large Language Models to systematically enhance test suite quality and maintainability.

🎯 Overview

This project extends the research work of Gay (2023) on "Improving the Readability of Generated Tests Using GPT-4 and ChatGPT Code Interpreter" by providing a fully automated, context-aware system for improving test readability. The system implements a sophisticated two-stage LLM process combined with MCP tools for comprehensive code analysis.

✨ Features

Core Capabilities

• Automated Test Improvement: Transform test suites without manual intervention
• Context-Aware Analysis: Deep understanding of source code and test relationships
• Six Transformation Types: Implements all readability transformations from research
• Quality Validation: Ensures syntactic correctness and PEP 8 compliance
• Batch Processing: Handle multiple test files simultaneously
• Comprehensive Logging: Complete transparency into the improvement process

Six Test Readability Transformations

1. Descriptive Test Names: test_case_0 → test_queue_initialization_with_valid_size
2. Documentation: Comprehensive docstrings with scenario descriptions
3. Clear Test Structure: Arrange-Act-Assert (AAA) pattern with explicit comments
4. Meaningful Variable Names: int_0 → max_size, queue_0 → queue
5. Informative Assertion Messages: Clear failure messages for better debugging
6. Code Organization: Logical grouping and improved structure

🚀 Quick Start

Prerequisites

• Python 3.13+
• OpenRouter API key (supports GPT-4o, Claude, etc.)

Installation

1. Clone the repository:

git clone <repository-url>
cd mcp-enhanced-testextender

2. Create and activate virtual environment:

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install --upgrade pip
pip install -r requirements_mcp.txt

4. Configure environment:

# Create .env file and add your API key
echo "OPENROUTER_API_KEY=your_api_key_here" > .env

Basic Usage

Run the complete demonstration:

python3 demo_real_mcp.py

Improve individual test files:

# Queue Example
python3 test_improvement_orchestrator.py \
  --source examples/queue_example/queue_example.py \
  --test examples/queue_example/test_queue_example.py \
  --output improved_tests

# HTTPie Sessions Example  
python3 test_improvement_orchestrator.py \
  --source examples/httpie_sessions/sessions.py \
  --test examples/httpie_sessions/test_httpie_sessions.py \
  --output improved_tests

# String Utils Validation Example
python3 test_improvement_orchestrator.py \
  --source examples/string_utils_validation/validation.py \
  --test examples/string_utils_validation/test_string_utils_validation.py \
  --output improved_tests

Programmatic usage:

from test_improvement_orchestrator import improve_single_test_file

result = await improve_single_test_file(
    source_file="examples/queue_example/queue_example.py",
    test_file="examples/queue_example/test_queue_example.py",
    output_dir="improved_tests"
)

if result.success:
    print(f"✅ Improved test saved to: {result.improved_filepath}")
else:
    print(f"❌ Error: {result.error}")

📁 Project Structure

mcp-enhanced-testextender/
├── examples/                          # Test case examples
│   ├── queue_example/
│   │   ├── queue_example.py          # Source: Queue data structure
│   │   └── test_queue_example.py     # Original tests
│   ├── httpie_sessions/
│   │   ├── sessions.py               # Source: HTTP session management
│   │   └── test_httpie_sessions.py   # Original tests
│   └── string_utils_validation/
│       ├── validation.py             # Source: String validation utilities
│       └── test_string_utils_validation.py # Original tests
├── improved_tests/                    # Generated improved test files
├── mcp_tools_server.py               # MCP tools implementation
├── openrouter_llm_client.py          # Two-stage LLM processing
├── test_improvement_orchestrator.py   # Main workflow coordinator
├── demo_real_mcp.py                  # Complete demonstration script
├── requirements_mcp.txt              # Project dependencies
└── README.md                         # This file

🔧 Core Components

MCP Tools Server (mcp_tools_server.py)

Six specialized tools for comprehensive code analysis:

• Source Code Reader: Parse and extract class/method information
• Test File Analyzer: Analyze test structure and quality metrics
• Context Discovery: Map relationships between source and test code
• Code Quality Checker: Validate syntax and identify improvement areas
• Prompt Generator: Create dynamic, context-specific instructions
• Test Transformer: Apply systematic transformations

Two-Stage LLM Processor (openrouter_llm_client.py)

• Stage 1: Generate context-aware improvement prompts
• Stage 2: Transform tests using generated instructions
• OpenRouter Integration: Support for multiple LLM providers

Test Improvement Orchestrator (test_improvement_orchestrator.py)

• Workflow Coordination: Manages the complete improvement process
• Quality Validation: Ensures correctness and compliance
• Output Generation: Creates improved tests, reports, and logs

📊 Results and Performance

Quantitative Improvements

Metric	Original	Improved
Generic Test Names	100%	0%
Missing Docstrings	100%	0%
Quality Issues	37 total	0
Syntactic Correctness	✅	✅

Processing Performance

• Average Processing Time: ~63 seconds per test file
• Token Efficiency: ~14,000 tokens per test file
• Success Rate: 100% (maintains all original functionality)

📋 Generated Outputs

Each improvement run generates:

1. Improved Test File: test_[original]_[example]_[timestamp].py
2. Improvement Report: improvement_report_[example]_[timestamp].md
3. Complete Prompt Log: all_prompts_[example]_[timestamp].txt

🛠️ Advanced Usage

Custom Model Selection

python3 test_improvement_orchestrator.py \
  --source examples/queue_example/queue_example.py \
  --test examples/queue_example/test_queue_example.py \
  --model "anthropic/claude-3-sonnet"

Focused Transformations

python3 test_improvement_orchestrator.py \
  --source examples/queue_example/queue_example.py \
  --test examples/queue_example/test_queue_example.py \
  --focus documentation  # Options: naming, documentation, structure, assertions, setup, all

Batch Processing

test_configurations = [
    {
        "source_file": "examples/queue_example/queue_example.py",
        "test_file": "examples/queue_example/test_queue_example.py"
    },
    {
        "source_file": "examples/httpie_sessions/sessions.py", 
        "test_file": "examples/httpie_sessions/test_httpie_sessions.py"
    }
]

results = await improve_multiple_test_files(test_configurations)

🧪 Example Transformation

Before (Original)

def test_case_0():
    int_0 = 1256
    queue_0 = module_0.Queue(int_0)
    assert queue_0.max == 1256
    assert queue_0.head == 0
    assert queue_0.tail == 0
    assert queue_0.size == 0

After (Improved)

def test_queue_initialization_with_valid_size():
    """
    Test the __init__ method of the Queue class.

    Scenario: Initialize a queue with a valid size.
    Expected Outcome: The queue is initialized with the correct attributes.
    """
    # Arrange
    max_size = 1256

    # Act
    queue = module_0.Queue(max_size)

    # Assert
    assert queue.max == max_size, "The max size should be set correctly."
    assert queue.head == 0, "The head should be initialized to 0."
    assert queue.tail == 0, "The tail should be initialized to 0."
    assert queue.size == 0, "The size should be initialized to 0."

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Gay, G. (2023): "Improving the Readability of Generated Tests Using GPT-4 and ChatGPT Code Interpreter" - Original research foundation
• Model Context Protocol (MCP): Standardized tool communication framework
• OpenRouter: Multi-provider LLM access platform

📚 Research Context

This project was developed as part of a Ph.D. application demonstration for the TestExtender project, focusing on automated test improvement and software engineering research. The implementation demonstrates practical applications of LLM-assisted software development tools with emphasis on code quality and maintainability.

---