Skip to content

[Phase 0.2.4] Create e2e test framework #10

@richard-devbot

Description

@richard-devbot

Phase

Phase 0 — Foundations | Track 0.2 — Test Infrastructure | Priority: P1

Summary

Create tests/e2e/ for full agent pipeline integration tests.

What

  • Create tests/e2e/__init__.py
  • Create tests/e2e/conftest.py with:
    • mock_llm_provider — deterministic LLM that returns scripted responses
    • test_agent — fully configured agent instance with mock LLM
    • test_gateway — gateway with mock channel for message injection
    • test_orchestrator — orchestrator wired to test agent and gateway
  • Create tests/e2e/helpers.py with:
    • send_message() — inject a message and capture full response pipeline
    • assert_tool_called() — verify specific tools were invoked
    • assert_response_contains() — check response content

Why

Unit tests verify individual components. E2E tests verify the full pipeline: message arrives → agent processes → tools execute → response sent. This catches integration bugs that unit tests miss.

Acceptance Criteria

  • tests/e2e/ directory exists with framework
  • Mock LLM provider returns deterministic responses
  • At least one e2e test: send message → receive response
  • Framework supports testing tool execution in pipeline
  • Tests run in CI without real LLM API calls

References

  • pytest-asyncio
  • Design Doc: docs/plans/2026-03-29-security-ai-guardrails-performance-design.md

Blocks

All Phase 4.2 issues (e2e tests use this framework)

Metadata

Metadata

Assignees

No one assigned

    Labels

    ProceedWork on the issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions