Benchmark Result Format

Benchmark Result Format

Purpose

This page defines the JSON artifact format for XAI OS benchmark results.

Requirements

Every performance result must include:

• XAI OS commit and build identity;
• hardware and firmware description;
• workload description;
• tuned Linux/BSD baseline description;
• raw metrics;
• hot-path counters;
• notes for thermal, power, firmware, and measurement caveats.

JSON Schema Example

{
  "benchmark": "tcp_latency",
  "platform": "intel_desktop",
  "hardware": {
    "system": "example desktop",
    "cpu": "example cpu",
    "memory": "2x32GB DDR5",
    "nic": "example nic",
    "storage": "example nvme",
    "firmware": "example firmware version"
  },
  "xaios_commit": "0000000000000000000000000000000000000000",
  "baseline": {
    "os": "linux",
    "kernel": "example",
    "tuning": ["isolcpus", "nohz_full", "irq_affinity"]
  },
  "workload": {
    "connections": 1000,
    "payload_bytes": 256,
    "duration_seconds": 60,
    "model": "example quantized cpu-only model",
    "agents": 1
  },
  "metrics": {
    "p50_us": 0,
    "p95_us": 0,
    "p99_us": 0,
    "p999_us": 0,
    "max_us": 0
  },
  "counters": {
    "core_migrations": 0,
    "context_switches": 0,
    "page_faults_after_ready": 0,
    "interrupts_on_hot_cores": 0,
    "timer_ticks_on_hot_cores": 0,
    "unexpected_ipis": 0
  },
  "notes": []
}

Validation Rules

• Do not publish a performance claim without a matching result artifact.
• QEMU on macOS results may validate correctness only.
• Use null for unavailable fields rather than deleting required keys.
• Keep raw logs alongside summarized JSON when possible.

Related Pages

• Testing and Benchmarking
• Performance Targets
• Risk Register