Testing Guide

Testing Guide

Test suite overview and best practices

---

Overview

Comprehensive test coverage:

• 107 test files
• 85%+ coverage
• Unit tests - Isolated logic
• Integration tests - API endpoints
• E2E tests - Full workflows

Framework: pytest

---

Quick Start

Run All Tests

# Activate environment
source .venv/bin/activate

# Run tests
pytest

# With coverage
pytest --cov=src --cov-report=html

# Parallel execution
pytest -n auto

Run Specific Tests

# Single file
pytest tests/test_chat.py

# Single test
pytest tests/test_chat.py::test_chat_completion

# By marker
pytest -m unit
pytest -m integration

---

Test Structure

tests/
├── conftest.py              # Fixtures and configuration
├── factories.py             # Test data factories
├── unit/                    # Unit tests
├── integration/             # Integration tests
├── routes/                  # API endpoint tests
├── services/                # Service layer tests
├── db/                      # Database tests
└── security/                # Security tests

---

Test Configuration

pytest.ini

[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
markers =
    unit: Unit tests
    integration: Integration tests
    slow: Slow-running tests
    critical: Critical path tests

Markers

@pytest.mark.unit
def test_function():
    pass

@pytest.mark.integration
def test_api_endpoint():
    pass

@pytest.mark.slow
def test_large_dataset():
    pass

---

Fixtures

Common Fixtures

# conftest.py

@pytest.fixture
def client():
    """FastAPI test client"""
    from src.main import app
    return TestClient(app)

@pytest.fixture
def test_user(db):
    """Create test user"""
    user = create_user(
        email="test@example.com",
        username="testuser"
    )
    yield user
    # Cleanup
    delete_user(user.id)

@pytest.fixture
def test_api_key(test_user):
    """Create test API key"""
    key, key_id = create_api_key(
        user_id=test_user.id,
        key_name="Test Key"
    )
    yield key
    delete_api_key(key_id)

Usage

def test_chat_completion(client, test_api_key):
    response = client.post(
        "/v1/chat/completions",
        headers={"Authorization": f"Bearer {test_api_key}"},
        json={
            "model": "gpt-3.5-turbo",
            "messages": [{"role": "user", "content": "Hello"}]
        }
    )
    assert response.status_code == 200

---

Test Examples

Unit Test

# tests/services/test_pricing.py

def test_calculate_cost():
    from src.services.pricing import calculate_cost

    cost = calculate_cost(
        input_tokens=1000,
        output_tokens=500,
        model="gpt-4"
    )

    assert cost == 0.065  # $5/1M input + $15/1M output

Integration Test

# tests/integration/test_chat.py

@pytest.mark.integration
async def test_chat_with_openrouter(client, test_api_key):
    response = client.post(
        "/v1/chat/completions",
        headers={"Authorization": f"Bearer {test_api_key}"},
        json={
            "model": "openai/gpt-3.5-turbo",
            "messages": [{"role": "user", "content": "Say hello"}]
        }
    )

    assert response.status_code == 200
    data = response.json()
    assert "choices" in data
    assert len(data["choices"]) > 0

Route Test

# tests/routes/test_users.py

def test_get_user_balance(client, test_api_key):
    response = client.get(
        "/user/balance",
        headers={"Authorization": f"Bearer {test_api_key}"}
    )

    assert response.status_code == 200
    data = response.json()
    assert "credits" in data
    assert isinstance(data["credits"], (int, float))

---

Mocking

Mock External APIs

from unittest.mock import AsyncMock, patch

@patch('src.services.openrouter_client.OpenRouterClient.chat_completion')
async def test_chat_with_mock(mock_chat, client, test_api_key):
    # Setup mock
    mock_chat.return_value = {
        "choices": [{
            "message": {"role": "assistant", "content": "Mocked response"}
        }]
    }

    # Make request
    response = client.post("/v1/chat/completions", ...)

    # Verify mock called
    assert mock_chat.called
    assert response.status_code == 200

Mock Database

@patch('src.config.supabase_config.get_supabase_client')
def test_with_mock_db(mock_supabase):
    mock_supabase.return_value.table.return_value.select.return_value.execute.return_value.data = [
        {"id": 1, "username": "test"}
    ]

    result = get_user(1)
    assert result["username"] == "test"

---

Coverage

Generate Report

pytest --cov=src --cov-report=html
open htmlcov/index.html

Coverage Goals

• Overall: >80%
• Critical paths: 100%
• Routes: >90%
• Services: >85%
• Utils: >90%

Exclude from Coverage

# pragma: no cover
def debug_function():  # pragma: no cover
    print("Debug only")

---

Continuous Integration

GitHub Actions

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.10'
      - run: pip install -r requirements.txt
      - run: pytest --cov=src

---

Database Testing

Setup Test DB

@pytest.fixture(scope="session")
def test_db():
    # Create test database
    db = create_test_database()
    yield db
    # Cleanup
    drop_test_database(db)

Transactions

@pytest.fixture
def db_transaction():
    # Start transaction
    transaction = db.begin()
    yield
    # Rollback
    transaction.rollback()

---

Best Practices

1. Isolate tests: No dependencies between tests
2. Use fixtures: Reuse setup code
3. Mock external calls: Don't hit real APIs
4. Test edge cases: Errors, limits, edge values
5. Keep tests fast: <1s per test
6. Clear assertions: One concept per test
7. Descriptive names: test_user_cannot_delete_others_api_key

---

Common Patterns

Parametrized Tests

@pytest.mark.parametrize("input,expected", [
    (1000, 0.001),
    (10000, 0.01),
    (100000, 0.1),
])
def test_credit_conversion(input, expected):
    result = cents_to_credits(input)
    assert result == expected

Async Tests

@pytest.mark.asyncio
async def test_async_function():
    result = await async_chat_completion(...)
    assert result is not None

---

Troubleshooting

Tests hang

Cause: Async operations not awaited

Solution: Use @pytest.mark.asyncio and await

Database conflicts

Cause: Tests not isolated

Solution: Use transactions or unique test data

Flaky tests

Cause: Race conditions, external dependencies

Solution: Mock external calls, add retries

---

Related Documentation

• Live API Testing - Test live APIs
• Failover Testing - Test failover

---

Last Updated: December 2024 Status: Active Development