Skip to content

Conformance matrix execution and benchmark CI #21

@FumingPower3925

Description

@FumingPower3925

Summary

Execute the full conformance suite (#13) across the complete engine × protocol matrix with full API integration, and establish benchmark regression tracking.

Conformance Matrix

7 engine configurations × 3 protocols = 21 combinations:

Engine Config HTTP/1.1 H2C Auto
io_uring Base
io_uring Mid
io_uring High
io_uring Optional
epoll
StdEngine
Adaptive

CI Benchmarks

Run all 5 benchmarks from #13 against every engine configuration:

  • BenchmarkPlaintext
  • BenchmarkJSON
  • BenchmarkPathParam
  • BenchmarkBodyRead
  • BenchmarkRealisticAPI

Regression Detection

  • Store benchmark baselines as CI artifacts
  • Compare against previous baseline on each PR
  • Fail CI if any benchmark regresses >10% in ns/op or allocs/op
  • SDD 14.5 performance regression benchmarks tracked as CI metrics

Deliverables

  • Matrix test runner executing all 21 combinations
  • Benchmark baseline storage and comparison
  • CI job with regression gate (>10% = failure)
  • Clear regression report in CI output

Tests

  • All 21 matrix combinations pass
  • Benchmark baselines stored correctly
  • Regression detection triggers on >10% degradation
  • CI output clearly identifies which benchmark regressed

SDD Reference

Sections 14.1, 14.5

Dependencies

Depends on #13, #19, #20

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions