feat: multi-turn implementation #232
Description
Working PR:
#226
Motivation and Use Cases
Why Multi-Turn?
Modern conversational AI systems like ChatGPT, Claude, and production chatbots operate in multi-turn mode where:
- Context matters: Each response depends on conversation history
- Sequential dependencies: Turn N+1 must wait for turn N's response
- Realistic workloads: Single-turn benchmarks don't reflect production patterns
- Performance characteristics differ: Multi-turn introduces blocking, queueing, and memory accumulation
High-Level Architecture
Component Diagram
┌─────────────────────────────────────────────────────────────────┐
│ Load Generator │
│ │
│ ┌───────────────────┐ ┌──────────────────┐ │
│ │ MultiTurnDataset │────▶│ ConversationMgr │ │
│ │ (JSONL loader) │ │ (State tracking)│ │
│ └───────────────────┘ └──────────────────┘ │
│ │ │ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────────────────┐ │
│ │ MultiTurnScheduler (Turn Sequencing) │ │
│ │ • Parallel mode (all turn-1s at t=0) │ │
│ │ • Sequential mode (complete conv by conv) │ │
│ │ • Poisson mode (arrival distribution) │ │
│ │ • Optional concurrency control │ │
│ └──────────────────────────────────────────────┘ │
│ │ │
└───────────┼──────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Endpoint Client │
│ Worker Process → HTTP Request → OpenAI Adapter │
│ (Forwards conversation_id + turn_number in metadata) │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Metrics Recorder │
│ EventRecorder: SQLite events.db with conversation fields │
│ MetricsReporter: Per-turn and per-conversation metrics │
└─────────────────────────────────────────────────────────────────┘