[E2E] Add Signal-Decision Engine Test Coverage for New Plugin Architecture #692
Description
Description
Add E2E test coverage for the signal-decision engine and plugin architecture introduced in #681. This complements #661 by testing the new decision-based routing mechanics.
Background
PR #681 introduced a breaking architectural change from category-based routing to a signal-decision engine with modular plugins.
Key concepts needing tests:
- Decisions: Priority-based routing with rules
- Plugins: Modular chains (PII → cache → system prompts)
- Signals: Input conditions (domain, keyword, entropy)
- Strategy: Decision selection (priority vs confidence)
Proposed Test Cases
1. Decision Priority & Strategy
- Multiple decisions matching → highest priority wins
- Confidence-based strategy vs priority-based
- Fallback to default when no match
2. Plugin Chain Execution
- Plugins execute in order
- PII plugin blocks entire chain when triggered
- Cache plugin works after PII passes
- System prompt applied after cache miss
3. Rule Condition Logic
- AND operator (all conditions must match)
- OR operator (any condition matches)
- Keyword matching (case-sensitive/insensitive)
- Domain + keyword combination
4. Plugin Configuration
- PII allowlist (
pii_types_allowed: ["PERSON"]) - Per-decision cache thresholds
- Plugin-specific configurations
5. Edge Cases
- No matching decision → fallback behavior
Example Test
func testDecisionPriority(ctx context.Context, client *kubernetes.Clientset, opts testcases.TestCaseOptions) error {
// Query matches both "thinking" (priority 15) and "business" (priority 10)
query := "Think carefully about this business decision"
response := sendChatRequest(opts.EnvoyService, query)
// Verify highest priority won
if response.Headers["X-VSR-Selected-Decision"] != "thinking_decision" {
return fmt.Errorf("expected thinking_decision, got %s", response.Headers["X-VSR-Selected-Decision"])
}
return nil
}
Implementation Plan
Phase 1: Core Logic
- Decision priority/strategy selection
- Rule conditions (AND/OR)
- Fallback behavior
Phase 2: Plugin Chains
- Execution order
- Blocking behavior
- Configuration variations
Phase 3: Integration
- Multi-signal routing
- Plugin interactions
- Error handling
Acceptance Criteria
- Decision priority and strategy selection tested
- Plugin chain execution order verified
- Rule condition logic (AND/OR) covered
- Plugin configuration variations tested
- Documentation updated in
e2e/README.md
Related Issues
- [Feat]: Signal-Decision Driven Semantic Routing with Dynamic Plugin Architecture #681 - Signal-Decision architecture PR (breaking changes)