-
Notifications
You must be signed in to change notification settings - Fork 1
Conformance matrix execution and benchmark CI #21
Copy link
Copy link
Closed
Labels
area/testTesting infrastructureTesting infrastructurephase/4-integrationPhase 4: Public API + DXPhase 4: Public API + DXplatform/portableCross-platform codeCross-platform codepriority/critical-pathBlocks other milestone workBlocks other milestone worksize/M~2-3 days of work~2-3 days of work
Milestone
Description
Summary
Execute the full conformance suite (#13) across the complete engine × protocol matrix with full API integration, and establish benchmark regression tracking.
Conformance Matrix
7 engine configurations × 3 protocols = 21 combinations:
| Engine Config | HTTP/1.1 | H2C | Auto |
|---|---|---|---|
| io_uring Base | ✓ | ✓ | ✓ |
| io_uring Mid | ✓ | ✓ | ✓ |
| io_uring High | ✓ | ✓ | ✓ |
| io_uring Optional | ✓ | ✓ | ✓ |
| epoll | ✓ | ✓ | ✓ |
| StdEngine | ✓ | ✓ | ✓ |
| Adaptive | ✓ | ✓ | ✓ |
CI Benchmarks
Run all 5 benchmarks from #13 against every engine configuration:
BenchmarkPlaintextBenchmarkJSONBenchmarkPathParamBenchmarkBodyReadBenchmarkRealisticAPI
Regression Detection
- Store benchmark baselines as CI artifacts
- Compare against previous baseline on each PR
- Fail CI if any benchmark regresses >10% in ns/op or allocs/op
- SDD 14.5 performance regression benchmarks tracked as CI metrics
Deliverables
- Matrix test runner executing all 21 combinations
- Benchmark baseline storage and comparison
- CI job with regression gate (>10% = failure)
- Clear regression report in CI output
Tests
- All 21 matrix combinations pass
- Benchmark baselines stored correctly
- Regression detection triggers on >10% degradation
- CI output clearly identifies which benchmark regressed
SDD Reference
Sections 14.1, 14.5
Dependencies
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/testTesting infrastructureTesting infrastructurephase/4-integrationPhase 4: Public API + DXPhase 4: Public API + DXplatform/portableCross-platform codeCross-platform codepriority/critical-pathBlocks other milestone workBlocks other milestone worksize/M~2-3 days of work~2-3 days of work